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1. 


Introduction. 


In  recent  years  there  has  been  an  explosive  growth  in  research 
dealing  with  the  development  and  complexity  analysis  of  combinatorial 
.ilgorithms.  While  much  of  this  research  is  theoretical  in  nature,  many 
of  the  newly  discovered  algorithms  are  very  practical.  These  algorithms 
ond  the  data  manipulation  techniques  they  use  are  valuable  in  both 
combinatorial  and  numeric  computing.  Some  problems  which  at  first 
glance  are  entirely  numeric  in  character  require  for  their  efficient 
solution  not  only  the  proper  numeric  techniques  but  also  the  proper  choice  of 
data  structures  and  of  data  manip\ilation  methods.  An  example  of  such  a 
I'roblem  is  the  solution  oi'  a system  of  linear  equations  when  the  coefficient 
matrix  contains  mostly  zeros  (Tewarson  [1975])* 

In  this  paper  I shall  survey  some  of  the  recent  results  on 
complexity  of  combinatorial  algorithms,  examine  some  of  the  ideas  behind 
them,  and  s\;iggest  possible  directions  for  future  research.  Section  2 of 
',.he  pa}*er  discusses  ways  to  measure  the  complexity 

of  algorithms.  Thoiigh  several  different  meas\ires  are  useful  in  different 
circumstances,  I shall  concentrate  upon  one  measxire,  the  worst-case 
running  time  of  the  algorithm  as  a function  of  the  input  size.  Section  3 
vliscusses  techniques  for  proving  that  certain  combinatorial  problems  are 
very  hard  to  solve.  The  results  in  this  area  are  a natural  ex+.ension, 
perhaps  more  relevant  for  reai-world  computing,  of  the  incompleteness 
;ind  undecidability  results  of  G^del,  Turing  and  others.  Section  t presents 
a small  collection  of  general  techniques  ;diich  are  usefiil 
in  the  construction  of  efficient  combinatorial  algorithms.  Section  5 
discusses  efficient  algorithms  for  solving  ten  representative  problems. 


These  problems  Illustrate  the  importance  of  the  methods  in  Section  and 


they  include  some,  but  certainly  not  all,  of  the  combinatorial  problems 
. for  which  good  algorithms  are  known.  Section  6 suggests  some  unsolved 

' problems  and  directions  for  future  research.  The  appendix  contains  a 

list  of  terminology  for  those  unfamiliar  -with  graph  theory. 


2.  Machine  Models  and  Comp.'Lexi ty  Measures. 

In  the  eeirly  years  of  computing  (before  computer  science  was 

recognizable  as  an  academic  discipline),  an  individual  confronted  with  a 

computational  problem  was  likely  to  proceed  in  the  following  way.  He  or 
» / 

she-^  would  ponder  the  problem  for  a vdiile,  formulate  an  algorithm  for 
its  solution,  and  write  a computer  program  wiiich  wjuld  hoj^efully  implement 
his  algor itlrn.  To  test  the  algorithm's  correctness,  he  would  run  the 
program  on  several  sets  of  data,  "debugging"  the  program  -until  it 
produced  correct  output  for  each  set  of  sample  input.  To  test  the 
algorithm's  efficiency,  he  would  measure  the  time  and  storage  space 
needed  by  his  program  to  pi'ocess  the  sample  data,  fit  these  measurements 
to  curves  (by  eye,  by  least-squares  fit,  or  by  some  other  method),  and 
claim  that  these  curves  measured  the  efficiency  of  the  algorithm. 

The  drawbacks  of  this  empirical  approach  are  obvious.  The  developjcent 
of  very  large  programs,  such  as  compilers  and  operating  systems,  requires 
a much  more  systematic  method  of  checking  correctness.  This  need  has  led 
comp'uter  scientists  to  devise  methods  for  ] roving  the  correctness  (ajid 
other  properties)  of  programs  (Floyd  [I967],  Manna  [19t'9}>  Hoare  [1969])- 
These  methods  use  mathematical  induction  to  establish  that  certain  invariant 
relations  hold  whenever  certain  points  in  the  progran  eLre  reached.  Computer 
scientists  iiave  also  developed  methods  (such  as  "structured  programming") 
for  constructing  easy-to-understand  and  easy-to-veri fy  programs  (Dahl, 
Dijkstra,  and  Hoare  [19,'2]),  and  have  formulated  new  programming  I'uiguages 
to  make  these  methods  easy  to  apply  (Wirth  [1971] The  thrust  of  this 
research  is  to  demonstrate  that  devising  an  algorithm  and  devising  a proof 
of  its  correctness  are  inseparable  p<arts  of  t.  _•  same  process.  Perhaps 
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Henceforth  I shall  use  "he" 


to  denote  any  individual. 


male  or  female. 
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the  i'oreinost  advocate  of  this  point  of  view  is  Dijkstra  (Dalil,  Dijkstra 
and  Hoare  [1972];  Dijkstra  [1976])- 

Measuring  efficiency  by  means  of  empirical  tests  has  the  same 
deficiency  as  checking  correctness  empirically;  there  is  no  guarantee 
that  the  result  is  reproducible  on  new  sets  of  data.  If  an  informed 
choice  is  to  be  made  between  two  algorithms  for  solving  the  same  problem,  some 
more  systematic  information  about  the  algorithms'  complexity  is  needed. 

To  be  most  useful,  this  information  should  be  machine- ‘.ndep  endent;  good 
algorithms  tend  to  remain  good  even  if  they  are  expressed  in  different 
programming  languages  or  run  on  different  machines.  Furthermore  the 
measure  should  be  both  realistic  and  susceptible  to  theoretical  study. 

Compilexity  m.easures  are  of  twc'  kinds : those  which  are 

statue  (independent  of  the  sine  and  characteristics  of  the  input  data) 
and  those  which  are  dp'namic  (dep)endent  upon  the  input  data'.  A typical 
static  measure  is  program  length.  Program  length  in  some  sense  measures 
the  simplicity  and  elegance  of  an  algorithm  (an  algorithm  with  a short 
program  and  short  correctness  proof  is  simple;  an  algorithm  with  a short 
program  and  long  correctness  proof  is  elegant).  This  measure  is  most 
appropriate  if  programming  time  is  important  or  if  the  program  is  to  be 
run  infrequently. 

Dynamic  complexity  measures  provide  infoimiation  about  the  resource 
requirements  of  the  algorithm  as  a function  of  the  characteristics  of 
the  input  data.  Typical  dynamdc  measures  are  running  time  and  storage 
space.  These  measures  are  appropriate  if  the  program  is  to  be  run  often. 
Running  time  is  usually  the  most  important  factor  restricting  the  size  of 
problems  v^ich  can  be  solved  by  computer;  most  of  the  problems  to  be 
examined  in  Section  5 require  only  linear  space  for  their  solution. 
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However,  for  problems  with  linear-time  algorithms,  storage  space  may 
be  the  limiting  factor.  Storage  space  has  Deen  used  as  a measure  in 
p-roofs  of  the  comp'Utational  intractability  of  certain  problems  (see 
Section  2),  but  most  efficiency  studies  emphasize  running  time. 

Dynainic  measures  reqiiire  that  we  sp-'ecit'y  the  input  data.  One 
possibility  is  to  assume  that  the  data  for  a given  problem  size  is  the 
worst  possible.  A worst-case  measure  of  running  time  or  storage  space 
as  a function  of  problem  size  provides  a performance  guarantee;  the 
program  will  always  require  no  more  time  or  space  than  that  specified 
by  the  bound.  A worst-case  measure  is  in  this  sense  not  unlike  a proof 
of  program  correctness. 

For  some  algorithms  a worst  case  bound  may  be  overly  pessimistic; 
for  instance,  the  simp'lex  method  of  linear  programming  (Dantzig  [19'"5l)> 
which  has  an  exponential  worst-case  time  bound  (Klee  and  Minty  [1972]), 
seems  to  run  much  faster  than  exponential  on  real-world  problems  (Dantzig 
[19t'5]).  In  such  cases  an  "average"  case  or  "representative"  case  may 
give  a more  realistic  bound.  For  certain  problem  domains,  such  as  sorting 
and  searching  (Knuth  [1973]),  average  case  analysis  is  almost  always  more 
realistic  than  worst-case  analysis,  and  in  these  areas  much  average-case 
analysis  has  been  done.  However,  average-case  analysis  has  its  drawbacks. 
It  may  be  very  hard  to  choose  a good  probability  measure.  For  instance, 
assuming  that  different  parts  of  the  input  data  are  independently 
distributed  may  make  the  analysis  easier  but  may  be  an  unrealistic 
assumption;  f\xrthermore  even  a relatively  simple  algorithm  may  rapidly 
destroy  the  independence.  With  average-case  analysis  one  additionally 
runs  the  risk  of  being  surprised  by  a very  rare  but  very  bad  set  of  input 


data. 


Any  concrete  coraj.lexity  neacure  riuct  be  bacecl  on  a computer  model.  One 


possible  choice  is  the  random  access  machine  (CojI:  and  r.eckr  [19^5]  )>  which 
is  an  abstraction  of'  a genei-al-purjose  digital  comyuter.  Tlie  .r.emory  of  such 
a machine  consists  of  an  array  of  storage  cells,  each  able  to  hold  an 
integer.  The  storage  cells  are  nuir.bered  consecutively  from  one;  the  nurler 
of  a storage  cell  is  its  address . The  machine  also  has  a I'ixed  finite  set 
of  registers,  each  able  to  hold  an  integer.  (For  problerr  involving  real 
numbers,  we  allow  storage  cells  and  registers  to  hold  real  nurrbers.  ) In 
one  step,  the  machine  can  transfer  the  contents  of  a register  to  a storage 
cell  ;^/hose  address  is  in  a register,  or  transfer  no  a register  the  contents 
of  a storage  cell  wiiose  address  is  in  a register,  or  pert  im.  an  arithmetic 
operation  on  the  contents  of  two  registers,  or  compare  tlie  contents  of  two 
registers.  A ; r gram  of  fixed  finite  length  specifies  tlie  sequence  of 
operations  to  be  carried  out.  Tiie  inital  configuration  of  memory 
represents  the  input  data,  and  the  final  configtu'ation  of  m.emory  represents 
the  output.  The  details  of  this  machine  model  tire  unirp'ortant  in  that 
reasonable  variations  do  not  affect  running  time  or  storage  space  by  more 
than  a constant  factor. 

A random  access  machine  is  sequential:  it  carries  out  one  step  at 
a timie.  Much  work  has  been  done  on  the  computati.inal  complexity  of 
parallel  algorithms,  but  I shall  not  discuss  this  work  here. 

The  random-access  machine  model  irovides  a useful  tool  for 
realistically  measuring  the  efficiency  of  particular  algorithms, 
but  it  has  serious  drawbacks  I'or  lower  bo\md  studies.  Gince  a 
single  storage  cell  can  hold  an  arbitrarily  large  integer,  it  is 
possible  on  a random  access  machine  to  carry  out  com.putations  in 
parallel  by  encoding  several  small  numbers  into  one  large  one.  One 


TUI  MV  ’id  this  j robleTn  by  assirning  that  the  time  reqa:r«-d  for  an  integer 
deration  is  -j  rojortional  to  the  length  of  its  binary  representation 
(Aiio,  Hojcroft,  and  Ulli:ian  [I97U]'!,  or  by  requiring  that  all  integers  be 
boiuided  in  absolute  value  by  some  constant  times  the  size  of  the  irq  ut  data. 

Random.-access  machines  are  extremely  powerful;  in  larticulfir,  they 
can  j'erfor!-’,  arithmetic  on  addresses.  This  ability  is  useful  for  rer  resent :.r 
multidimensional  arrays  (Knuth  [19<'-‘']i,  performing  radix  .'ort:  frinuth  [I'-  '- 
storing  hash  tables  (Knuth  [lStT5]''j  the  like.  However,  determining  the 

theoretical  limits  of  this  caj  ability  seems  to  be  a iiard  j roblem. 

Kolmogorov  [1955]j  Kolmogorov  and  Usjenskii  [19<-"'5J)  Knuth  [19'o8], 

■IchHnhage  [197^)],  and  Tarjan  [1977]  have  proposed  machine  models  in  wirich 
access  to  memory  is  by  exi-licit  reference  only,  and  no  address  arithmetic 
is  possible.  I shall  call  such  a machine  a linked  memor-,  mach'ne.  These 
::.achines  accurately  model  the  capabilities  of  list-processing  languages 
such  as  LISI  and  the  list-p  rocessing  feat-ores  of  general-]  i;ri  nse  languages 
such  as  Alg  1-Vi  and  IL/1,  and  they  appear  to  be  m.ore  amenable  to  analysis 
than  random-access  m.achines. 

Another  ver^'  simple  machine  model,  the  Turing  machine  (Turing  [lo^'  -7]'. 
has  been  used  in  many  theoretical  studios.  A Turing  m.achine  iias  a mem.cry 
consisting  of  a tape.  The  tape  is  divided  into  cells,  each  capable  of 
holding  one  oi’  a finite  number  of  symbols.  The  machine  p -sses.'es 
a finite  internal  memorp/  and  a read/write  head  wiiich  can  scan  one  tape 
cell  at  a time.  In  one  stepi,  the  machine  can  read  a tape  cell,  write  a 
new  symbol  in  the  cell  (erasing  what  was  there  pre-</-iously),  m.-.ve  the 
read/write  head  one  cell  forward  or  backward  on  the  tap  e,  and  change 
the  internal  memory  state.  The  decision  as  to  what  t do  at  each  step 
depiends  only  on  the  current  internal  memory  state  and  the  contents  of 


the  tape  cell  being  read;  this  decision  i£ 


encoded  for  each  internal 
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jtate  and  eacii  tape  syiabol  in  a decision  table  which  forns  the  program 
oi  the  machine. 

'Puring  proposed  his  machine  model  in  1936,  before  electronj c digital 
compaters  existed;  he  was  attempting  to  model  comj'utatlonaJ.  processes  in 
the  abstract,  without  rei'er^'nce  to  any  real  computer.  Though  Turing' s 
"'.odel  is  inadequate  for  a large  p^art  of  concrete  complexity  research,  its 
simplicity  and  the  fact  that  any  random  access  machine  can  be  simulated 
on  a Turing  machine  with  only  a polynomial  blow-up  in  running  time  makes 
the  Turing  machine  extremiely  useful  for  studying  very  difficult  com.]  utatior.ai 
tasks.  It  is  :ilso  valuable  for  studying  problems  where  tapes  are  the 
stoi'age  device,  as  for  Instance  in  tape  sorting  (Khuth  [1973]  )• 

In  lower  bound  studies  the  focus  is  often  on  some  critical  operation; 
one  counts  in  the  running  tim.e  occurrences  only  of  that  critical  operation. 

For  instance,  in  sorting  and  selection  problems  it  is  useful  to  count  -nly 
comparisons  (or  general  binar;,’  decisions),  measuring  the  complexity  of  a 
] roblem  by  the  depth  of  a decision  tree  for  it  (Aho,  Hopcrcft,  and  Ullman 
[lSi7^])-  In  arithmetic  an  i algebraic  problems,  it  is  useful  to  count  only 
arithmetic  operations  and  to  assume  that  no  decisi.'ns  are  m.ade;  i.e.,  that 
the  cor.putations  p erfonru.-d  are  independent  of  the  input  data  (tlr  a parfculai’ 
problem  size).  In  this  case  one  measures  the  comp  le.xity  d'  a p roblemi  by  the 
length  of  a straight-line  prograja  (Aiio,  Fiopcroft,  and  ’jlLman  In 

other  situations  memory  accesses  may  be  the  critical  op’eratien^ . 

In  this  paper  I shall  use  worst-case  runn’.ng  time  on  a random-access 
machine  as  a measure  of  algorithmic  comip  lexity.  This  raeasuj’e  is  useful 
and  realistic  for  a wide  range  of  c.mbinatorial  problems.  I slirtll 
ignore  constant  factors  in  ruiming  time,  since  such  constant  factors  depend 
upon  the  exact  model  of  computation,  they  are  often  hard  to  compute,  and 


P 
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they  tend,  at  least  for  large- sized  problems,  to  be  washed  out  by 
asymptotic  growth  rates.  To  indicate  fvinctional  relationships,  I 
shall  use  the  following  notation.  If  f and  g are  functions  of  n , 

" f(n)  is  0(g(n))  " means  f(n)  < cg(n)  for  all  n , wiiere  c is  a 
suitable  positive  constants  and  ” f(n)  is  Q(g(n))  " means  f(n)  _■  cg(n) 

for  all  n , where  c is  a suitable  positive  constant. 


Complexity  of  Intractable  Problems. 

Inspired  by  Hilbert  [1926]  and  other  formalists,  mathematicians  of  the 
early  twentieth  century  hoped  to  find  a formal  system  vAiich  would  be  adequate 
for  expressing  and  veryifying  all  mathem.atical  truths.  These  hopes  were 
dashed  by  Gttdel  [1931]>  w?io  in  his  famous  incompleteness  theorem  demonstrated 
that  no  method  of  proof  could  be  both  subject  to  mechanical  verification 
and  powerful  enough  to  prove  all  theorems  of  elementary  arithmetic.  Their 
interest  in  the  foundations  of  mathematics  prompted  logicians  to  ccni'ront 
the  q’uestion,  "What  is  mechanical  verification?"  or  equivalently,  "ii/hat  is 
an  algorithm*?".  Church  [195'  Kieene  [1936],  lost  [1936],  Turing  [l,‘3'-7]'> 
and  others  provided  form.al  definitions  of  an  algorithm.  These  definitions 
are  superficially  different  but  jrovably  equivalent,  in  the  sense  that  if 
a problem  is  solvable  according  to  one  definition  of  an  algorithm,  then 
it  is  solvable  according  to  all  the  other  definitions.  This  robustness 
of  the  notion  of  an  algorithm,  is  usually  stated  as  Church’s  thesis:  any 

algorithm  (in  the  informal  sense)  can  be  expressed  as  a Turing  machine, 
and  any  Turing  machine  exj^resses  an  algorithm. 

Once  a formal  definition  of  an  algorithm  existed,  it  was  possible 
for  mathematicians  to  study  the  power  of  com.putation,  Turing  proved  that 
no  algorithm  existed  for  determining  wiiether  a given  Turing  machine  with  a 
given  input  will  ever  halt.  Other  researchers  discovered  a number  of  such 
undecidable  problems  (Jones  [197^]),  which  correspond  in  computer  science 
to  the  incomp'leteness  results  of  Gttdel  and  others  in  logic.  lerhaps  the 
capstone  to  this  research  on  computability  is  Matijasevic' s l-''70  proof, 
building  on  earlier  work  by  Martin  Davis  and  Julia  Robinson,  that  Hilbert's 
tenth  problem  is  undecidable  (Davis,  Matijasevic,  and  Robinson  [197'- )'• 
Hilbert's  tenth  problem  is  to  determine  whether  a given  polynomial  equation 
has  a solution  in  integers. 


Two  proof  techniques,  di agonali zatlon  and  simulation,  jervade 
computability  theory.  Diagonalization  is  based  on  ancient  seli'-i-eference 
paradoxes;  Cantor  [I87I]  used  it  to  prove  that  there  are  re  real  number, 
integers  and  Gtidel  used  it  to  prove  his  incompletenecs  result.  Cue  can 
use  it  in  the  following  way  to  devise  an  undecidable  probler..  Ciajpose 
wc  are  interested  in  yes-no  questions  about  the  integers,  such  as  "]s  n 
even?"  or  "Is  n prime?"  Suppose  we  have  a listing  A^.k.A,, ...  of 
all  algorithms  for  answering  such  questions  (for  any  of  the  standard 
definitions  of  an  algorithm  it  is  easy  to  produce  such  a listing). 

Consider  the  set  S of  integers  such  tliat  n is  an  element  oi‘  f if 
and  only  ii'  algorithm  answers  "no"  (or  does  not  .answer  at  all)  on 

input  n . Then  the  question  "Is  n an  element  of  S ?”  is  undecidable, 
since  each  algorithm  in  the  list  A^,A„,A-,...  1 reduces  a vn'-ng 

answer  on  at  least  one  injut  ( A^^  is  wrong  on  input  n ) and  by  Church's 
thesis  this  list  contains  all  possible  algorithms.  Tuidng  used  the 
same  idea  to  show  the  undecidability  of  the  halting  problem  for  Turing 
machines. 

Simulation  is  a method  for  turning  one  problem  or  problem-solving 
method  into  another.  Once  we  have  one  undecidable  problem  , we  can 

prove  another  problem  l’^  undecidable  by  showing  that  if  has  an 

algorithm  then  this  algorithm  can  be  used  to  solve  F^j^  . To  accomr'lish 
this  we  provide  an  algorithm  which  converts  an  instance  -f  problem. 
into  one  or  more  instances  of  problem  Fg  , thus  reducing  F^  to  F^ 

(or  transforming  into  P^  ).  Similarly,  to  show  that  two  definitions 

of  an  algorithm  are  eqxzivalent,  we  show  how  to  simulate  an  algorithm 
according  to  one  definition  by  an  algorithm  according  to  the  other 
definition. 


that 
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The  develoi'ment  of  genercil-puri'ose  digital  computers  made  possible 
the  implementation  and  execution  of  complicated  algorithms,  and  the 
theory  of  computability  became  a matter  of  more  than  mathematical 
intei'est.  However,  this  theory  ignores  questions  of  resource  use,  which 
limits  its  power  to  identity  what  is  possible  in  jractice.  Many  problems 
wliicli  obviously  liave  algor ittrcs  seem  to  have  no  good  algo>'ithm".  For 
instance,  consider  the  maximum,  stable  set  problem:  given  a graph,  find  in  it 

a maximum  number  of  vertices,  no  two  adjacent.  Since  a graph  with  n 
vertices  has  only  2^  subsets  of  vertices,  an  exponential-time  algorithm, 
for  this  j r blem.  exists.  However  n.  one  has  yet  discovered  a substantially 
faster  algorithm  for  th.I  s p rcblem.. 

[Table  '.Ij 

Tables  ’.1  a:id  3-  ' illustrate  t!ie  irrp 'rta.nce  ol‘  this  phenomenon. 

Table  ^.1  ■ - Imates  rurcilng  times  of  algor' tlur.s  -with  vari  us  ticr.e 
bounds.  The  table  show'  that  constant  factors  become  less  and  less 
important  as  problem,  sice  increases;  on  large  problems  the  asymj'totic 
growth  rate  of  the  time  bound  dominates  the  constant  factor.  The  table 
also  shows  that  running  time  grows  explosively  if  the  time  bound  is 
exponential.  Table  3.2  estimates  the  maximum  sise  of  p'roblemc  solvable 
in  a given  amount  of  tim.e.  Increasing  the  amount  of  time  (or  the  speed 
of  the  machine)  by  a large  factor  does  not  substantially'  increase  the 
size  of  problems  solvable  unless  the  time  bound  grows  more  slowly  than 
exponential. 

[Table  ’ . ] 

Tables  5.I  and  3.2  sxiggest  a naiural  division  between  g. 'Od  algsrithm.r 
(those  with  worst-case  time  bounds  polynomial  in  the  size  of  the  inp-.ut) 
and  bad  algorithms.  Edmonds  [19' 5]  was  app'arently  the  first  to  stress  this 
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distinction.  I shall  call  a decidable  problem  tractai)le  if  it  has  a 


polynomi al-time  algorithm  and  intractable  otherwise.  The  didinction 
'V  between  tractable  and  intractable  problems  is  independent  of  Uie  machine 

model,  since  any  of  the  commonly  used  machine  models  can  be  simulated  by 
any  other  with  only  a polynomial  loss  in  running  time.  As  Tables  1 and  2 
show,  it  is  not  feasible  to  execute  exponent! al-time  algorithms  on  large 
j'.roblems.  Miiny  combinatorial  problems  are  easily  solvable  in  exponential 
time  by  exhaustively  checking  cases,  but  solving  such  problems  in  j.olynomial 
time  seems  to  require  much  greater  insi^it.  Most  known  good  alg  rithms 
have  time  bounds  wliich  are  jolynomi als  of  small  degree  ( 0(n'^)  or  better). 

It  is  a major  task  of  complexity  theory  to  identify  wiiich  natura.  pr  blems 
are  tractable  and  which  are  intractable. 

i 

Hartmanis,  Lewis,  and  Stearns  took  the  first  steps  toward  exi.ibiting  ! 

j 

natural  intractable  problems  (Hartmanis,  Lewis,  euid  Stearns  [19o5l; 

Hartmanis  and  Stearns  [I965]).  By  diagonalizing  over  all  algoritiims  with 
a given  spjace  bound  S^(n)  , they  were  able  to  obtain  a problem  solvable 

in  space  S2(n)  but  not  in  space  S^(n')  , for  any  spjace  bounds  S^(n)  and  i 

S^Cn)  satisfying  lim  inf  S^(n')/S2(n)  = 0 and  a few  other  tedinical 

n -•<» 

constraints.  They  proved  a similar  but  somewhat  weaker  result  for  j 

time  complexity.  These  results  imply  in  particular  that  there  are  problems  ! 

solvable  in  exponential  space  but  not  in  polynomial  space,  and  problems  ’ 

I 

solvable  in  exponential  time  but  not  in  polynomial  time. 

Uhfortiinately,  the  intractable  problems  produced  by  diagonal! zation 
are  not  natural  ones.  Meyer  and  Stockmeyer  [1972]  proved  the 
intractability  of  a natural  problem.  They  showed  that  the  problem  of 

determining  whether  two  regular  exp'ressions  with  squaring  denote  the  j 
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same  set  requires  exi'onential  space  (and  hence  exp'onential  time) 
for  its  solution.  A regular  expression  is  a formula  constructed  from 
the  symbols  A,0,1,U>  "j  (>  ) according  to  the  follo-/rIng 

rules.  Each  such  formula  denotes  a set  of  strings  of  zeros  and  ones. 

5.1  0 is  a regular  exp-ression  denoting  the  set  {0}  ; 

1 is  a regular  exj:ression  denoting  the  set  {1]  : 

A is  a regular  expression  denoting  the  set  whose  single  element 

is  the  empty  string. 

3.2  If  A and  B are  regular  expressions  denoting  sets  L(a)  and 

L(B)  , respectively,  then 

(AUB)  is  a regular  expression  denoting  the  set  L(A)uL(B) 

(A*B)  is  a regular  expression  denoting  the  set 
(xy  ) X r L(A)  and  yeL(B)}  . 

A*  is  a regular  expression  denoting  the  set  consisting 
oi  the  empty  string  and  all  strings  formed  by  concatenating 
one  or  more  strings  in  L(A)  . 

Meyer  and  Stockmeyer  added  an  additional  rule; 

2 

5.3  If  A is  a regular  expression,  then  A is  a regular  expression 

denoting  the  same  set  as  (A*A)  . 

To  prove  that  the  equivalence  problem  for  two  such  express’ one  is 
intractible,  Meyer  and  Stockmeyer  used  simulation.  They  devised  a 
polynomial -time  algorithm  which,  given  a Turing  machine,  an  inp'ut,  and 
an  exponential  space  bound,  woxild  construct  a regular  expression 
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—’j  i’'^--eutln'K  the  computation  of  the  Turing  machine  on  tin-  giv- " 'n^ut. 

The  •.:X'jiresc i on  is  such  that  it  denotes  the  same  set  -as  ( l - 

if  aiii  only  if  the  Turing  machine  does  not  accej-t  the  iinut  vlthin  the 

given  sj  ace  bound.  It  follows  that  the  equivalence  ] robl-r::  f r regular 

ressions  with  squaring  is  as  hard  (to  within  a rolyn  >r.i  rjl  time  blov;-ui  ) 

as  yes-no  question  answerable  in  exj-onential  si  ace  by  a lliring  machine. 

.:ince  the  Hartmani-,  Lewis,  Stearns  result  inq.lies  that  some  j roblem 

n 

t.-xist-^  which  can  be  solved  in  sjace  2‘  but  not  si  ace  -'n  • the 
oquivfilence  jroblem  for  regular  ex]ressions  must  re-quire  cxqoncntial 
. i ace. 

Tn  the  last  five  years,  several  more  such  results  hav--  h-ien 
1’ scovered.  Hunt  [I?;?]  showed  that  if  set  Intersec"  i s.  i.  : st ' t stt-d  " v 
squ;iring  the  equivalence  jroblem.  for  regular  ex^ressiuns  st'll  requires 
■xi.'unential  sjac--.  St.ycteieyes  and  I-'.eyer  [1 'i’-' 1 .li  ws-d  ti'a' 
subtraction  is-  substituted  for  squaring  the  equivalence  ]r.bl-r  for 
regular  expressions  has  a non-elementaii/  sj-ace  bound.  Fisch-er  .-uid  ’-abin 
[I'l-'^-J  1 roved  that  testing  the  validity  :f  a for:.ula  in  ; r-  .rg-.-r 
arithmetic  (the  theory  of  natural  nambers  with  + as  the  only  ojeratisn'' 

gcn 

requires  2 space,  for  some  positive  constant  c . Carlrsa,  Hi  ton. 

and  Meyer  [I'JT’  l showed  that  the  word  jroblem  for  Abelian  gr  uj  . iv.-quires 
-ixionential  sqace.  Jaz'ayeri,  Ogden,  and  Rounds  [l'/('5l 
showed  that  testing  the  circularity  of  attribute  gra-mr.ars  (a  qr-blem 
arising  in  programitLing  language  semantics)  requires  exi-onenti al  tira-. 

The  idea  in  all  these  proofs  is  the  same;  one  shows  hew  to  efficient.ly 
convert  any  computation  with  a jarticular  space  or  time  carjlexity  intr 
an  instance  of  the  given  problem,  and  one  appeals  to  the  Harir.ani..  L*'wis, 
Steams  results  to  assert  the  exi stence  of  an  intractible  i roblem  with 


th<’  ! artlc’olai-  :ica  -v  time  eoini'lexity. 

Significantly,  a nu.mlei-  of  aj'parently  intractable  jroblen.j,  cucli  ac 
the  niaximutr.  stable  set  problem,  are  not  Included  in  the  list  of  known 
Intractable  in'oblems.  These  T^robleins  have  the  following  irperty.  If 
such  a I I’oblem  is  phrased  as  a yes-no  question,  and  tlie  ;oiswer  s "yes", 
then  there  is  a polynomial- length  j roof  of  the  answer.  For  instance, 
sup'pose  we  rephrase  t'ne  naxlma’n  stable  set  problem  as  follows:  "Doe., 

a given  graph  G contain  a stable  set  of  k vertices?"  If  the  answer 
is  yes,  one  can  prove  it  by  exhibiting  the  stable  set  .-and  showing  tliat 
its  vertices  are  j.air;’.ise  nen-adjacent. 

To  f- nsalise  this  notion  of  ] ol^aionitl- length  proof,  we  'ntrrduce 
non-deteim.lni  sti c machine^  , A non-deterraLnisti  c machine  -.a;,',  at  varl  >u. 
times  during  its  coirq  utation,  nal'.e  a guess  as  tj  what  t.  do  next.  The 
machine  accepts  a given  imut  if  there  exists  some  sequence  guesses 
which  Causes  the  .-nachine  to  eventually  answer  "yes".  We  define  tlie  time 
(or  space)  required  by  the  machine  to  accept  an  input  as  the  irinimur. 
ar.ount  of  time  (or  sj  ace  used  by  an  accejting  commutation.  The  following 
non-determini  Stic  algor' tlua  solves  the  maximum  stable  set  problem  in 
polynomial  time:  First,  guess  a subset  of  k vertices.  Next,  check  all 

pairs  jf  these  vertices  r adjacency.  Accept  if  no  two  of  the  vertices 
'ire  adjacent.  Let  denote  the  class  of  yes-no  j'robleras  solvable 
deterministically  in  p.lynomial  time  and  let  Tjp  denote  the  class  of 
yes-r'.o  pr  iolems  solvable  non-detenninistically  in  } lynomial  ti.-.e.  The 
question  we  wish  to  answer  is,  "Are  there  natural  pToblems  which  :ire  in 
but  not  in  o ?" 

Cook  sh -wed  that  719  contains  certain  "hardest"  problems, 

called  9 -complete  p roblems . A problem  I is  7(p -complete  if 
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it  satisi’ies  two  properties; 

5 . t r i s in  TIP  . 

5.5  If  Q is  in  7(p  then  Q is  reducible  to  T in  pclj’nonijtl  time. 

To  s£iy  that  Q is  reducible  to  F in  j'olynoraial  time  meaiac  that 
there  is  a (deterministic)  polynomial- time  algorithm  wiiich,  g-veti  an 
instance  of  ircblem  Q.  , will  convert  it  into  an  inct;ince  oi’  ;r  blem  F , 
such  that  the  answer  to  the  instance  of  Q is  "ye. " if  and  only  if  the 
answer  to  the  instance  of  F is  "yes".  If  Q is  reducible  to  F in 
polynomial  time  and  F has  a polynomial- time  algorithm,  then  so  does  Q . 
Thus  if  any  V;- p -complete  problem  has  a polynomial -time  algorithm.,  p = T(p  . 

Cook's  main  result  was  to  show  that  the  satisfiability^  j roblem  of 
propositional  calculus  is  7? p -complete.  Hie  satisfiability  j roblem  is  to 
determine  whether  a given  logical  formula  is  true  for  at  least  one 
assignment  of  the  values  "true"  and  "false"  to  the  variables.  It  is  easy 
to  show  that  this  jroblem  satisfies  3**‘"  Cook  prirved  5.5  by  giving  a 
polynomial-time  algorithm  for  constructing,  from  a given  non-deterministic 
Turing  machine,  a given  input,  and  a given  polynomial  time  bound,  a logical 
form.ula  such  that  the  formiila  is  satisfiable  if  and  only  if  the  Turing 
machine  accepts  the  input  within  the  timie  bound. 

If  one  knows  a single  problem  F to  be  -comjdete,  one  can  prove 

another  problem  Q 7? p -complete  by  showing  that  Q is  in  TiP  and  that  F 
is  reducible  in  polynomial  time  to  Q ; property  5*5  then  follows  from  the 
transitivity  of  polynomial -time  reducibility.  Karp'  [197’C]  used  this 
idea  to  exhibit  a number  of  natural  7iP -complete  problems.  Others 
continued  this  work,  and  the  number  of  known  7^p -complete  problems  is 
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U'.w  In  the  hundreds  (see  for  instance  iiven,  Itai,  and  Shamir 

Jarey,  Johnson,  and  Stockmeyei'  [V^(‘  J;  Garey,  Johnson,  and  Tarjan  [197u]; 

Kar;  [1G75]:  Sahni  [197^];  Sethi  [1975];  and  Ullman  [1975] )•  In  addition 
to  the  satisfiability  problem  and  the  maximum  stable  set  problem,  the 
following  problems  are  -complete. 

Subgraph  isomorihism  (Cook  [1971]).  Given  two  graphs  and  G^  , is 

G^^  icomonhic  to  a subgraph  of  G.-,  ? 

Graph  coloring  (Karp  [I972]).  Given  a graph  G , can  its  vertices  be 

colored  with  k colors  so  that  no  two  adjacent  vertices  have  the  sane 
color?  Tills  problem  is  '/(  j^ -complete  even  if  k = 3 a:id  G is 
planar  (Garey,  Johnson,  and  Stoctaneyer  [197'-]'',  whereas  it  follows 
from  Appel  and  Haken's  proof  of  the  four  color  conjecture  (An  el  and 
Haken  [ir/y])  than  there  is  a polynomial-time  algorithm;  to  color  any 
planar  graph  with  four  colors. 

Hardlton  cycle  (Karp,  [1972]).  Given  a graph  G does  it  contain  a cycle 
'^iiich  passes  through  every  vertex  exactly  once?  This  problem  is  a 
special  case  of  the  travelling  salesman  problem  (see  Section  1).  It 
is  7/ p -complete  even  if  G is  planar  (Garey,  Johnson,  and  Tarjan  [197o]) 

Subset  sum  (Karj  [I972]).  Given  a set  of  numbers  n^^, n.,, . . . , n,^^  and  a sum.  s , 
does  some  subset  of  the  numbers  sum  to  exactly  s ? 

Maximum  planar  subgrajh  (Liu  and  Geldmacher  [197'-’]'.  Given  a graph  G , does 
it  contain  a p'lanar  subgraph  with  at  least  k edges? 

A major  open  problem  of  com}  lexity  theory  is  to  deteimdne  wiieth-.r  p = 7,  p 
A natural  approach  to  t)ii  s problem  would  be  to  try  using  di 'jgonalisat  ■ n ! 
exhibit  a problem  in  7[p  but  not  in  p . However,  recent  work  by  i<aker. 

Gill,  and  Solovay  [1975]  suggests  that  diagonalisation  is  impotent  for 


resolving  the  p = 7?P?  question,  [•h/’en  without  a proof  that  p = rjp, 
it  is  still  fruitful  to  add  new  natural  problems  to  the  list  of 
72 P -complete  ones;  the  large  amount  of  time  spent  by  bright  people 
fruitlessly  searching  for  polynomial -time  algorithms  for  ?2P -complete 
problems  is  strong  evidence  that  the  T(p -complete  problems  are  in  fact 
intractable. 


i* . Tech!ii,|ie.;  for  Gjod  Algorithms. 

Although  many  important  combinatorial  problems  seem  to  be  intractable, 
many  others  have  good  algorithms.  A small  number  of  data  manipulation 
techniques  form  the  basis  I'or  these  algorithms.  This  section  examines 
these  techniques,  which  are  outlined  in  Table  1.1. 

[Table  1.1] 


Data  Structures. 

Any  algorithm  (good  or  bad'  requires  one  or  more  data  structures  to 
represent  the  elements  of  the  problem  to  be  solved  and  the  information 
computed  during  the  solution  process.  A data  structure  is  a composite 
object  composed  of  elements  related  in  specified  ways.  Associated  with 
the  data  structure  is  a set  of  operations  for  manipulating  its  elements. 
Once  a good  implementation  of  a given  data  structure  and  Its  operations 
is  known,  one  can  regard  the  operations  as  primitives  wtien  implementing 
any  algorithm  which  uses  the  data  structure.  The  efficiency  of  the 
algorithm  will  depend  to  a large  extent  upon  the  implementation  of  the 
underlying  data  structure. 

There  are  two  lata  structures  upon  which  all  others  are  based: 
arrap's  arid  linked  structures.  An  £trray  is  a collection  of  storage 
cells  n'ombered  consecutively.  Two  operations  are  associated  with  an 
array,  given  the  namber  of  a storage  cell,  one  can  either  store  a va.ue 
in  the  storage  cell  (destroying  the  current  va.ue)  or  retrieve  the  current 
value  i'rom  the  storage  cell.  The  memory  of  a random  access  machine  aiid 
f most  digital  computers  is  an  array.  One  cem  use  arrays  t rej resent 
vectors,  matrices,  tensors,  and  multidimensional  arrays  (Knuth  [19''^]'. 
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A linked  structui'e  consists  of  a collect'  .n  ''  rec  rd.  . 


n record 


is  divided  into  a number  of  items,  each  with  an  identii’ying  na.;;e.  T'ne 
stnicture  of  all  records  "c  identical.  Items  are  l'  iw  k'^d.  . lat a 
i tt:ms  and  reference  items.  Data  items  contain  data.  Reference  'terns 
■ontain  x^inters  to  records.  Two  o^'erations  are  jossible  on  a ' inked 
stiizctiire;  given  a X'’'inter  to  a record,  one  can  either  store  a value 
Inl  ' an  it-.'m  in  ill--  record  or  retrieve  the  current  value  from  an  item 
in  the-  rec  ;rd.  Figure  1.1  illustrates  a linked  structure.  Whereas 
acra./  addresses  are  integers  cajable  of  being  manipulated  by  arithmetic 
orerations.  no  operations  are  allowed  on  linked  stmeture  ( anf.-Ts 
except  storage,  retrieval,  and  testing  for  equality.  Tne  mem:u-y  f a 
i inked  memory  machine  is  a linked  stimicture,  and  most  list- 
I r ’Cesslng  lan^suages  can  be  regarded  as  operating  on  linked  structures. 

[Figzzre  l.ll 

It  is  easy  to  imx'.lement  arrays  and  linked  structures  so  that 
st.orage  and  retrieval  require  constant  time.  Linked  structures  can  be 
iraxlemented  as  collections  of  arrays  (see  Figure  1.1);  this  makes  lizt- 
processing  easy  in  languages  such  as  FORTRAN  which  do  not  losses^  ai\ 
explicit  list-processing  facility.  It  seems  to  be  imjiossible  to  implement 
an  arra;/  as  a linked  stmeture  in  such  a way  that  storage  and  retrieval 
take  constant  time,  though  I know  of  no  proof  of  this  fact. 

Using  arrays  and  linked  structures,  one  can  implement,  many  different 
data  structures.  I shall  consider  here  five  classes  of  data  structures: 
lists,  unordered  sets,  ordered  sets,  graphs,  and  trees, 

A list  is  a sequence  of  elements.  The  first  ele;nent  -f  a Mst  is 
its  head;  the  last  element  is  its  tail.  Simple  operations  'n  a list 
include  scanning  the  list  to  retrieve  its  elements  in  order,  adding  an 
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element  as  the  new  head  of  the  list  (making  the  old  head  the  second 
element);  adiing  an  element  as  the  new  tail,  deleting  and  retrieving 
the  head  of  a list,  and  deleting  and  retrieving  the  tail  of  a list. 

Lists  :>n  -vhich  only  a few  of  these  operations  are  possible  have  special- 
names . A stack  is  a list  with  addition  and  deletion  allowed  only  at  the 
head,  A queue  is  a list  with  addition  allowed  only  at  the  tai  and 
deletion  allowed  only  at  the  head.  A deque  (double-ended  queue)  is  a 
list  on  which  addition  or  deletion  is  possible  at  either  end.  One  can 
implement  a deque  either  as  a circular  array  (addresses  are  computed 
modulo  the  size  of  the  array)  or  as  a singly  linked  structure  (if  deletion 
from  the  tail  is  not  necessary''.  See  Figure  1,2.  In  either  case,  all 
operations  except  scanning  require  constant  time.  The  array  represent at io, 
uses  no  space  for  storing  pointers  out  requires  that  an  am.ount  of  storage 
equal  to  the  maximum  size  of  the  list  be  permanently  allocated  to  the  : i st 

[Figure  1.2] 

Other  important  list  operations  include  concatenating  two  lists 
(making  the  head  of  the  second  list  the  element  following  the  tail  of 
the  first),  inserting  an  element  before  or  after  an  element  whose  location 
in  the  list  is  known,  and  deleting  an  element  idiose  location  in  the  list 
is  known.  These  operations  require  a linked  structiire  for  t'leir  efficient 
implementation.  A singly  linked  structure  is  sufficient  for  concatenation 
and  for  insertion  after  another  element.  Insertion  before  another  element 
and  deletion  require  a doubly  linked  structure.  See  Figure  U.3.  An 
alternate  way  to  handle  deletion  is  to  provide  each  element  with  a flag 
which  is  set  to  "true"  if  the  element  is  to  be  deleted.  The  element  is 
not  explicitly  deleted  uiitil  the  next  scan  through  the  list. 


[Figure  I, 5] 


The  list  operations  hardest  to  implement  are  inserting  an  element 


at  the  k-th  position  in  a list,  retrel ving  the  element  at  the  k-th  position 
in  a list,  or  deleting  the  element  at  the  k-th  position  in  a list.  It  is 
possible  to  implement  these  operations  to  run  in  O(log  n)  time,  idiere  n 
is  the  size  of  the  list,  by  using  AVL  trees  (Knuth  [1975])  or  2-3  trees 
(Aho,  Hopcroft,  and  Ullman  [197^1 )>  which  are  rather  complicated  linked 
structures.  Recently  Guibas,  McCreight,  Plass,  and  Roberts  [197?]  have 
found  a way  to  carry  out  these  operations  in  O(log  k)  time. 

An  unordered  set  is  a co3_Lection  of  distinct  elements  with  no  imposed 
relationship.  Basic  set  operations  are  adding  an  element  to  a set, 
deleting  an  element  from  a set,  and  testing  ^diether  an  element  is  in  a 
set.  One  way  to  represent  a set  is  by  a singly  linked  list.  Addition 
requires  constant  time  but  testing  and  deli  tion  require  0(n)  time,  where 
n is  the  size  of  the  set.  Alternatively,  if  the  elements  of  the  set  are 
values  which  can  be  compared  and  sorted,  one  can  represent  the  set  by  an 
AVL  tree  or  a 2-5  tree  in  such  a way  that  all  three  operations  require 
O(log  n)  time  (Knuth  [1973];  Aho,  Hopcroft,  and  Ullman  [I97I]). 

Another  way  to  represent  a set  is  by  a bit  vector  (Aho,  Hopcroft,  and 
'Jllman  [197*+]  >diich  is  an  array  with  one  storage  cell  for  each  possible 
element.  A storage  cell  has  two  possible  values;  true,  indicating  that 
the  set  contains  the  element,  and  false,  indicating  that  it  does  not.  All 
three  operations  require  constant  time  using  this  representation.  Bit  vector 
representation  is  only  feasible  if  the  number  of  possible  elements  is  small. 

If  the  number  of  possible  elements  is  large,  one  can  mimic  the  behavior 
of  a bit  vector  by  using  a hash  table  (Knuth  [1973])*  A hash  table  consists 
of  a moderately  sized  array  and  a hashing  function  which  maps  each  possible 
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element  into  an  ai’ray  addrecc.  If  an  element  is  j;resent,  the  element 

(or  a pointer  to  it)  is  stored  at  (or  near)  the  address  sjjeclfied  by  the 
hashing  function.  Since  two  or  more  elements  may  hash  to  the  same  address, 
some  mechanism  must  be  provided  for  resolving  such  collisions.  Hash  tables 
are  used  extensively  in  compilers,  and  many  papsers  have  been  written  about 
them  (see  Knuth  [1975],  Morris  [1968]),  With  a hash  table,  addition, 
deletion,  and  testing  require  0(n)  time  in  the  worst  case  but  only 
constant  time  on  the  average. 

Additional  set  oj'erations  are  useful  if  two  or  more  sets  exist.  These 
include  the  ability  to  form  a set  which  is  the  union,  intersection,  or 
difference  of  two  sets.  For  most  rep-resentations  union,  intersection, 
and  difference  req'uire  time  proportional  to  the  sum  of  the  sises  oi'  the 
sets.  However,  if  the  -miverse  of  elements  is  small  enougli  so  that  a bit 
vector  can  fit  into  a few  computer  words  and  the  comp'uter  possesses  bit 
vector  operations,  then  union,  intersection,  and  difference  require  constant 
time. 

An  ordered  set  is  a collection  ol'  elements,  each  with  an  associated 
numeric  value.  Two  important  operations  on  ordered  sets  are  sorting  tiie 
elements  in  increasing  order  and  selecting  the  element  with  k-th  largest 
value.  A variety  of  ways  exist  to  sort  n elements  in  0(n  log  n)  time 
(Knuth  [1975]);  if  binary  comparisons  are  the  only  operations  used  to 
manipulate  the  values  then  Q(n  log  n)  time  is  reqviired  in  both  the  average 
and  the  worst  case  to  sort  (Knuth  [1973])*  Selecting  the  k-th  largest 
element  requires  0(n)  time  (Blum,  Floyd,  Pratt,  Pivest,  and  Tarjan  [19^'3]: 
SchBnhage,  Paterson,  and  Pippenger  [1575]). 

A priority  queue  is  an  ordered  set  on  wiiich  the  following  operations 
axe  allowed:  adding  an  element  to  the  queue,  retreiving  the  minimum-value 


element  in  the  queue,  and  deleting  an  element  whose  location  is  known 
from  the  queue.  By  using  binomial  trees  (Vuillemin  [1977]>  Brown  [1977]), 

>,  leftist  trees  (Knuth  [1973 ])j  or  2-5  trees  (Aho,  Hopcroft,  and  Ullman 

[197*+])  one  can  implement  priority  queue  operations  so  that  they  require 
O(log  n)  time,  vdiere  n is  the  size  of  the  queue.  These  implementations 
also  allow  one  to  combine  two  queues  into  a larger  queue  (destroying  the 
smaller  queues^  in  O(log  n)  time. 

If  the  values  of  the  elements  in  an  ordered  set  are  integers  of 
moderate  size,  then  the  ordered  set  operations  can  be  speeded  up.  Using 
a k-pass  radix  sort,  one  can  sort  n integers  in  the  range  1 to  m. 
in  0(km+n)  time  (Knuth  [1975])>  Peter  van  Erode  Boas  has  devised  a 
method  for  implem.ent ing  priority  queues  with  integer  values  in  the  range 
1 to  n so  that  the  queue  operations  require  O(log  log  n)  time 
(van  Emde  Boas,  Kaas,  and  Zijkstra  [1975]). 

A graph  is  a set  of  vertices  and  a set  of  edges,  each  edge  a pair 
of  vertices,  (iie  way  to  represent  a graph  is  by  a two-dimensional  array  A , 
called  an  adjacency  matrix.  The  value  of  A(i, j)  is  one  if  (i,J)  is  an 
edge  of  the  graph;  otherwise  the  value  of  A(i,j)  is  zero.  An  alternate 
way  to  represent  a graph  is  by  an  adjacency  structure,  which  is  aji  array 
of  lists,  one  for  each  vertex.  The  list  for  vertex  i contains  vertex  J 
if  and  only  if  (i,J)  is  an  edge  of  the  graph.  See  Figure  l.t. 

[Figiire  U.U] 

The  adjacency  matrix  representation  saves  space  if  the  graph  is  dense 
(i.e.,  most  possible  edges  are  present);  it  also  allows  one  to  test  the 
presence  of  a given  edge  in  constant  time.  However,  Anderaa  and  Rosenburg 
conjectured  (Rosenberg  [19731)  ar+d  Rivest  and  V\iillemin  [1975]  proved  that 

testing  any  non-trivlal  monotonic—'  graph  property  requires  Q(n  ) 

» / ^ 

A graph  property  is  non-trivial  if  for  any  n the  property  is  true  for 
some  graph  of  n vertices  and  false  for  some  other  graph  of  n vertices. 
A graph  property  is  monotone  if  adding  edges  to  a graph  does  not  change 
the  property  from  true  to  (‘alse. 
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probt'P  of  the  adjacency  tnatrix  in  the  worst  case,  where  n is  the  number 
vertices  in  the  graj'h.  By  using  ■jn  adjacency  structure,  one  can  search 
a graph  in  0(n+m)  time,  where  m is  the  number  of  edges  in  the  graph; 
thus  representation  by  an  adjacency  structure  is  preferable  for  sparse  grapJis, 
A tree  is  a graph  without  cycles.  Since  a tree  is  a grap^h  it  can  be 
represented  by  an  adjacency  structure.  A more  compiact  way  to  represent 
a tree  is  to  choose  a r/ot  for  the  tree,  compute  the  p'arent  of  each  verte.x 
with  respect  to  this  root,  and  store  tiiis  information  in  an  .'irray 
(Figure  U.5').  Hais  representation  is  usable  as  long  as  the  tree  is  to 
be  exp'lored  from  leaves  to  root,  winch  is  often  the  case  in  problems 
involving  trees. 

[Figure  1.5] 


Recursion. 

An  imp'ortant  and  very  general  algorithmic  technique  is  recursion. 
Recursion  is  a method  of  solving  a problem  by  reducing  it  to  one  or  more 
subproblems.  The  subproblems  are  reduced  in  the  same  way.  Eventually 
the  subproblems  become  sr.all  enou^  that  they  can  be  solved  directly.  Tlie 
solutions  to  bhe  smaller  subprobleras  are  then  combined  to  give  solutions 
to  the  bigger  subproblems,  until  the  solution  to  the  original  problem  is 
computed.  As  a simple  example  of  a recursive  algorithm,  consider  the 
following  definition  of  the  n-th  Fibonacci  namber; 


l.l  F(n)  ;=  if  (n=l)  or  (n  = 2)  then  1 else  F(n-l)  + F(n-.' 


Using  recursion,  one  can  often  state  algorithms  much  re  s;m; ly 
than  would  be  possible  without  recursion.  Many  programming  Imigu-wp  ■ , 
including  Algol,  PL/1,  and  LISP,  allow  recursive  procedures  (ir-co.iure. 
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wh;ioli  call  themselves).  In  a language  without  this  facility,  such  as 
FORTRAN,  one  can  Implement  a recursive  algorithm  by  using  a stack  to 
store  the  generated  subproblems  (Alao,  Hopcrol’t,  and  Ullman  [197^]). 

L^i^namic  programming  (Bellman  [195?])  can  be  viewed  as  a sj'ecial  kind 
of  recursion  in  which  one  keeps  track  of  the  generated  subproblems  and 
never  solves  the  same  problem  twice.  As  an  example  of  the  work  wtiich  can 
be  saved  in  this  Wiiy,  consider  the  computation  of  the  n-th  Fibonacci 
number.  A recursive  jrocedure  based  on  1.1  requires  time  proport i.nal 
to  the  size  of  F(n)  to  corap'Ute  F(n)  ; such  a procedure  performs 
F(rHl-i)  computations  of  F(i)  for  each  i in  the  rang--  from,  1 to  n . 

A better  way  to  compute  F(n)  is  to  compute  each  F(i)  .just  once  for 
each  value  of  i . The  most  efficient  way  to  implement  a dynamic  programming 
algorithm  is  to  set  up  a table  of  solutions  to  all  subproblems,  and  to  fill 
in  the  table  from  smallest  to  largest  subproblem.  Goraetimes  one  can 
discard  the  solutions  for  small  subproblems  as  tlie  computation  proceeds 
and  re-use  the  space  for  larger  subproblems.  One  can  evaluate  F(n)  in 
0(n)  time  with  two  storage  locations  by  using  this  idea.  (Of  course, 
using  a closed-form,  expression  for  F(n')  results  in  an  even  faster 
conputation.  ) 

hynamlc  programming  has  been  used  with  great  success  on  a number  of 
combinatorial  problem.s,  inciud.ng  short-est  path  problems  (Floyd  [19c2]), 
context-free  language  parsing  (Younger  [19*'7]>  Farley  [1970]'',  error 
correction  in  context-free  languages  (Aho  and  Peterson  [l'772l\  and 
construction  of  optimum  binary  search  trees  (Knuth  [1971]>  Itai  [197c'])- 

Graph  Searching. 

Most  graph  problems  require  for  their  solution  a system.atic  method 
of  exploring  a graph.  A search  is  an  examination  of  the  edges  of  a graph 


using  the  following  procedure. 
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.'t-':  1 (irJ  •.j.-xlLzation  : Mark  all  edges  and  vertices  of  the  graph  new 

(•ji.ex]  lured  . 

jte;!  2 (choose  a new  starting  vertex);  If  no  new  vertex  exists,  halt, 
(The  entire  graph  has  been  explored. ) Otherwise,  choose  a new 
vertex  and  nark  it  old  (explored). 

Gte;  (ex];lore  an  ed^e);  If  no  new  edges  lead  away  from  old  vertices, 
go  to  step  r>.  (All  .'f  the  graph  reachable  from  the  current  start 
vertex  has  been  ex;  lored, ) Otherwise,  choose  a new  edge  leading 
away  fr.-'m  an  old  vertex.  Mark  the  edge  old.  If  the  other  endpoint 
of  the  edge  is  new,  mcirk  it  old.  Repeat  step  5- 

Assume  for  simplicity  that  all  vertices  in  the  graph  to  be  searched 
•jre  reachable  from  the  first  start  vertex  selected  in  step'  2.  Then  the 
search  generates  a spanning  tree.  Hie  root  of  the  spanning  tree  is  the 
st-art  vertex.  The  edges  of  the  spanning  tree  are  the  edges  which  lead  to 
new  vertices  when  explored  in  step  5*  The  properties  of  the  spanning  tree 
depend  upon  the  criteria  used  to  select  the  starting  vertex  in  step  2 and 
the  edges  to  explore  in  step  5.  For  some  simple  graph  problems,  such  as 
finding  connected  components  (Hopcroft  and  Tarjan  [1973c]),  any  order  of 
exploration  is  satisfactory.  However,  for  harder  graph  problems  the 
exploration  order  is  crucial. 

In  a depth-first  search,  the  edge  selected  in  step  5 is  an  edge  out 
of  the  last  explored  vertex  with  candidate  edges.  If  a depth-first  search 
is  performed  on  an  undirected  graph,  the  generated  spanning  tree  has  the 
property  that  -qi  1 non- tree  edges  connect  vertices  related  in  the  tree 
(Tarjan  [ISfJ'S]).  See  Figure  1,6.  If  such  a search  is  performed  on  a 
directed  grapih  and  the  vertices  are  numbered  from  1 to  n as  they  are 
marked  old,  then  no  non-tree  edge  leads  from  a vertex  to  a vertex  wtiich  is 
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both  higher  niunbered  and  unrelated  in  the  spanning  tree  (Tru'.'-ui  [I-",  ]'. 
See  Figure  h.J.  A deptli-l'irst  search  can  be  imilenented  as  a rec’irs've 
procediire  or  with  an  explicit  stack  to  store  the  old  vertio-s. 

[ Figure  1 . ] 

[fiigure 

In  a breadtli- first  search,  the  tsdge  selected  in  ste]  !s  u.  elg>- 
out  of  the  first  explored  vertex  wM'n  candidate  eugt  . . Such  a s-  n’ch 
pru’titions  the  vertices  into  levels  de]  ending  U}^'n  their  distune-  fr -r:. 
the  st;iTt  vertex.  In  an  undirecte-i  grn]  h "--ch  edge  .unnects  has  h; 

the  same  level  or  in  two  adjacent  levels;  in  a directed  gmih,  n ' eug<, 
leads  from  a level  to  a leve]  higher  thun  the  next  level.  .’.ee  Figur-  •■ . . 
A breadtli- first  search  can  be  imidencn't  ed  using  a queue  to  sl.u-e  tin-  1: 
vertices. 

[Figure  1.  ] 

Both  depth-first  and  breadth-first  search,  if  ] ro] erly  imidenented 
using  an  adjacency  structure  to  store  the  grajh,  reou're  f)(ntm''  time  t i 
explore  an  n-vertex,  m-edge  grajh.  Althouj^i  these  are  the  most  imjsrtant 
search  methods,  several  others,  including  topological  search  (Knuth  [lo- 
lexicographic  seai-ch  (Geth!  [l  h’51;  Hose.  TarJ.-in,  and  Lucker  [l  ie  T:,  and 
shortest- first  search  (Dljkstra  [I'-.^Jp],  Johnson  [ lor-7  ] ' , are  occasionally 
useful. 


Cgdimlzation  Methods. 

A large  class  of  problems  requires  the  maximization  of  a function 
defined  on  a graph  with  weiglit  ed  ed^jes.  It  is  usu.olly  j ossible  t } hrase 
these  problems  as  linear  or  integer  programming  ] roblems  (Druitzig  [l  -’-'^), 
Nemhauser  and  Garfinkel  [l<r(’2n,  but  bettor  algorithms  than  general -j 'unose 


linear  or  integer  programming  methodij  are  available  for  their  solution. 


These  algoritlims  use  two  teciiniques,  greed  and  augmentation.  The  most 
general  setting  for  these  techniques  is  in  matroid  theory  (Lawler  [I976]), 
but  one  can  understand  and  apjdy  the  teciiniques  to  graph  problems  without 
knowing  about  matroids. 

Consider  the  problem  of  finding,  in  a set  with  weighted  elements,  a 
ma.'c.imum-weight  subset  satistying  certain  additional  constraints.  The 
following  greeay  method  migiit  be  useliil  in  solving  this  problem.  Sort 
the  elements  by  weight.  hbcamine  the  elements  in  order,  heaviest  to 
lightest,  builtUng  uj  a subset  element-by-element.  Wlien  examining  an 
element,  add  it  to  the  subset  if  S'.ne  extension  of  the  subset  satisfies 
the  constraint.  Otherwise  t.!;r  w the  element  away'.  Tlie  resultant  subset 
certainly  satisfies  th-’  ■■.■lu-traint.  'Jhie-  apjropriate  conditions,  the 
subset  will  be  of  maximum  j.^Lsible  weigjit.  One  problem  to  wtiich  this 
method  is  applicable  is  the  minimum  sj anning  tree  problem  (Kruskol  [1956], 
iTim  [1957],  Dijkstra  [l<''59l)  Yao  [l'775l>  Clieriton  and  Tarjan  [I976]). 
k>en  if  the  greedy  metliod  does  not  p roduce  optimal  solutions,  it  may 
produce  solutions  which  are  close  to  optimal  (Garey  and  Johnson  [I976]), 
and  it  is  usually  easy  to  implement  and  fast. 

In  situations  vrtiere  the  greedy  method  doesn't  work,  a method  of 
iterative  improvement  sometimes  does.  The  idea  is  to  start  with  any 
solution  to  the  constraints  and  look  for  a way  to  augment  the  weigiit  of 
the  solution  by  making  local  changes.  The  new  solution  is  then  improved 
in  the  same  way,  and  the  process  is  continued  until  no  irajirovement  is 
possible,  under  appropriate  conditions  such  a locally  maximal  solution 
is  also  globally  maxiimmi,  LVen  if  the  solution  is  not  guaranteed  to  be 

maximum,  the  augmentation  method  may  be  a good  heuristic;  for  instance,  # 
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Lin  fl9''5]  ha£^  aprlied  it  wltli  good  results  to  the  travelling  salesraai'. 
problem.  Tlie  travelling  salecm.-in  p'roblem  is  to  find  a shortest  cycle 
through  all  vertices  of  a graj’h  witii  distances  on  the  edges.  The 
Hamilton  cycle  problem,  a sj'ccial  case  of  the  travelling  salesman  problem., 
is  TiP  -coir.j  lete. 

Data  Upidating  Methods. 

Gome  problems  require  more  sophisticated  data  manipulation  than  is 
possible  with  the  simple  data  structures  discussed  early  in  this  section. 
Three  advanced  techniques  have  been  devised  for  dealing  with  three  diverse 
problems  which  require  dynamic  updating  of  data.  These  techniques  are 
] ath  compression,  p.'artition  refinement,  and  linear  arrangement. 

Path  compression  is  a method  of  solving  the  following  problem. 
Consider  a universe  of  elements,  partitioned  initially  into  singleton 
sets.  Associated  witii  each  element  is  a value.  We  wish  to  be  able  to 
carrj'  out  the  following  operations  on  the  sets. 

Union:  Combine  two  sets  into  a single  set,  destroying  the  old 

sets. 

Update : Modify  the  values  of  all  elements  in  a given  set  in  a 

consistent  way, 

l.Valuate : Retrieve  the  value  associated  with  a given  element. 

A situation  of  this  kind  occurs  in  the  compilation  of  FORTRAN  COMMON  and 
EQUIVALKNCE  statements  (Oaller  and  Fischer  [I96I])  and  in  several  other 
combinatorial  problems  (Tar j an  [1975b] The  set  union  problem  to  be 
discussed  in  Section  5 is  the  simplest  such  problem.  Caller  and  Fischer 
[1961+]  proposed  an  algorithm  for  this  problem  using  trees  as  a data 
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ctructure,  Mcllroy  and  Morris  confronted  the  set  union  jrobleni  when  trying 
to  compute  minimum  spanning  trees  and  proposed  an  improved  method  using 
path  compression  on  trees  (Aho,  Hopcroft,  and  UlJman  [19’i^U]).  Their  method, 
which  is  very  simple  to  program  but  very  hard  to  analyze,  generalizes  to  a 
number  of  otiier  problems  (Tarjan  [1975b]).  I shall  discuss  this  method 
and  its  remarkable  running  time  in  Section  5. 

Another  pu’oblem  involving  disjoint  sets  is  the  following.  Suppose 
the  vertices  of  a graph  are  initiallj'  p'artitioned  into  several  subsets. 

We  wish  to  find  the  coarsest  partition  which  is  a refinement  of  the  given 
partition  and  which  is  preserved  \mder  adjacency,  in  the  sense  that  if  two 
vertices  v and  w are  contained  in  the  same  subset  of  the  p;irtition, 
then  the  sets  A(v)  = {x  | (v, x]  is  an  edge]  and  A(w)  = [x  | (w, x)  is  an  edge! 
intersect  exactly  the  same  number  of  times  with  each  subset  of  the  partition. 

Tills  adjacency-preserving  p.artition  is  easily  computable  in  0(nm)  time. 

Hopcroft  [1971]  devised  a more  sophisticated  algorithm  which  runs  in 
0(m  log  n)  time.  Gries  [19751  Si'^^s  a nice  description  of  this  algorithm. 

Partition  refinement  is  useful  in  solving  the  state  minimization  problem 
for  finite  automata  (Harrison  [I965])  and  in  testing  graphs  for  isomorphism 
(Cornell  and  Gottlieb  [1970]'i. 

A third  problem  requiring  a good  data  updating  method  is  the  linear  j 

arrangement  problem:  Given  a set  of  n elements  and  a collection  of  | 

subsets  of  the  elements,  can  the  elements  be  arranged  in  a line  so  that 
each  subset  occurs  contiguously?  This  problem  arises  in  biochemistry 
(Benzer  [1959])  and  in  archaeology  (Kendall  [19^9]).  Booth  and  Lueker 
[1976]  have  devised  a method  of  solving  this  problem  in  o(n+m)  time, 

where  m is  the  total  size  of  the  subsets,  using  a data  struct\ire  they  1 

i 

call  a F-Q  tree,  I 
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Tlaere  are  two  methods  of  solving  graph  problems,  decomposition  and 
shrinking,  viiich  are  related  to  the  algebraic  concepts  of  sub algebra  and 
homomorphism.  One  way  to  solve  certain  graph  problems  is  to  decompose  the 
graph  into  several  subgraphs,  solve  the  problem  on  the  subgraphs,  and 
combine  the  solutions  to  give  the  solution  for  the  entire  graph.  In  most 
instances  wliere  this  technique  is  useful,  the  subgrap^hs  are  components 
(maximal  subgraphs)  satisfying  some  connectivity  relation.  In  order  to 
apply  the  technique,  one  must  know  on  efficient  wa;;,"  to  determine  the 
Ci'mponents,  Good  algorithms  exist  for  a variety  of  connectivity  problems 
(Tarjan  [1972],  Hoporoft  and  Tarjan  [1973a],  Hopcrolt  and  Tarjan  [1975c], 
lacault  [197^]>  Tarjan  [197*+a],  Tarjan  [1975c]'l, 

Another  way  to  solve  some  graph  problems  is  to  shrink  part  of  the  graph 
to  a single  vertex,  solve  the  problem  on  the  shrunken  graph  by  applying 
the  idea  recursively,  and  from  this  solution  compute  the  solution  on  the 
original  graph.  The  shrinking  operation  corresponds  to  taking  a homomorphic 
image  of  the  grap^h.  Generally  the  part  of  the  graph  to  be  shrunk  is  a 
cycle  or  a union  of  cycles. 


Ten  Tractable  Iroblems. 


P . 

Hiere  ai’e  hundreds  of  combinatorial  problems  for  which  good  algorithms 
are  known.  This  section  examines  ten  such  problems.  I have  selected  the 
i robloms  on  the  basis  of  their  importance,  the  range  of  techniques  they 
require,  and  ::'y  familiarity  with  them,  Tlie  list  is  not  meant  t^’  be  exliauctive 
but  to  be  representative  of  problems  with  good  algorithms.  Table  5,1 
lists  the  problems  and  the  techniques  used  in  the  best  algorithms  for  the!.. 
Figure  5,1  shows  improvements  in  s lution  time  achieved  recently  for 
these  problems, 

[Table  5.1] 

[Figure  5.1] 


Discrete  Fourier  Tr;uisl~orT::. 

Given  an  n-dimencional  vector  (a^,  a., , . . , , ' , the  discrete  Fourier 

transform  problem  is  to  comp'Ute  the  vector  (b^,  b^,  . , . , b^  given  by 

b,  = 23  • wtiere  are  the  (comrlex')  n-th  roots  of 

i = 0 

•r.e.  This  p-r  hlem  arises  in  signal  processing.  An  algorithm  for  the 
discrete  Fourier  transform  is  useful  as  a subroutine  in  various  arithmetic 
and  algebraic  problems,  including  polynomial  evaluation  and  interpolation 
aiid  integer  and  polynomial,  multiplication  (Knuth  [19<j9]>  Alio,  Hopcroft, 
and  Ullman  [197l]j  Borodin  and  Munro  [1975]'', 

It  is  strai^tforward  to  compute  the  discrete  Fourier  transform  in 

2 

0(n  ) time.  Cooley  and  Tukey  [19^5]  popularized  an  0(n  log  n)  -time 
method,  called  the  fact  Fourier  tr,'msform.  They  were  not  the  first  to  use 
the  method,  which  originated  at  least  as  early  as  Runge  and  KBnig  [192li], 


The  fast  Fourier  transform,  uses  recursion  to  cut  down  the  amount  of 


k 


ccjnj.'utation.  Recently  Winograd  flS75,  197&]  ]. reposed  a method  for 
computing  the  discrete  Fourier  transform  using  only  0(n)  multiplications. 
This  method  may  be  sup-erior  to  the  fact  Fourier  transform  in  practice, 
although  Winograd  has  not  analyzed  the  overall  running  time  of  his 
algorithm. 

Matrix  Multiplication. 

Given  two  nxn  matrices,  the  matrix  multiplication  problem  is  to 

determine  their  matrix  piroduct.  The  standard  high  school  method  of 

matrix  multiplication  requires  O(n^)  time.  Strassen  [I969]  devised  a 

way  to  multiply  two  2x2  matrices  with  only  seven  multipdications,  and 

used  this  in  a recursive  matrix  mxiltiplication  algorithm  requiring  only 
log^  7 

0(n  ^ ) time.  This  surp'rising  result  has  acted  as  a stimulus  for 

mcuh  research  in  the  complexity  of  aiLgebraic  problems.  No  one  knows 
whether  Strassen' s algorithm  is  improvable.  Strassen' s algorithm  has 
been  used  to  compute  transitive  closures  of  graphs  (Munro  [1971],  Fischer 
and  Meyer  [I97I])  and  to  do  context-free  language  parsing  (Valiant  [1975a]) 
in  0(n^'®^)  time. 

A problem  related  to  matrix  multiplication  is  the  shortest  path 
problem.  Given  a directed  graph  with  positive  edge  distances,  the  single 
source  shortest  path  problem  is  to  find  the  minimum  distance  from  a given 
vertex  to  every  other  vertex.  The  all  piai rs  shortest  path  problem  is  to 
find  the  minimum  distance  between  all  pairs  of  vertices.  Dijkctra  [195'"*] 
devised  an  algorithm  for  the  single  source  problem  which  requires  either 
0(n  ) time  or  0(m  log  n)  time  depending  upon  the  imrlementati -n,  wiiere 
n is  the  number  of  vertices  and  m the  number  of  edges  in  the  graph 
(Johnson  [1977])*  Floyd  [19^2]  gave  a way  of  solving  the  all  j'airs  problem 
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In  0(rr  ' 


Lline.  Fredinan  [iy7h]  showed  that  the  all  pairs  j r^blern  can 

'''5  ^ l/^ 

be  solved  using  o(n-'"^)  comparisons  and  only  0(n'  (log  log  n/log  n)  ''  ) 

time  total.  Avis,  Rivest,  smd  Yao  [1977]  prwed  thiit  at  least  fi(n  log  n^ 

comparisons  are  required  in  the  wr'rst  case  to  solve  the  all  jairs  jroblem. 

This  lower  bound  is  one  of  the  few  known  for  a tractable  combinatLU-ial 

problem. 

Linear  Equations  on  a llanai'  Graih. 

Suppose  A is  an  nxn  matrix,  b is  an  nvl  vector  ol'  constants. 
X is  an  nyl  vector  of  variables,  jmd  we  -wisVi  t;  solve  the  system  of 
equations  Ax  = b , A standard  method  for  doing  this  is  daussian 
eliminati-n  (Forsythe  and  Moler  [loi.Y],  Tewarson  [l'['7^]  'i.  First,  the 
matrix  A is  decomtiosed  into  a jroduct  of  two  matris-..,  A = L'J  . such 
that  L is  lower  triangular  (i.e.,  L has  no  non-zero  entries  above  the 
diagonal)  and  U is  iq  ler  triangular  (i.e.,  U has  no  nun-zerc.-  entries 
below  the  diagonal).  Then  Ax  = b is  solved  in  two  steps,  by  solving 
Ly  = b , called  front  solving,  jind  solving  Ux  = y , called  backsolvinr. 
Because  L and  U have  special  forms,  frontsolving  and  backsolving  are 
very  efficient;  the  slowest  pjirt.  of  Gaussian  elimination  is  the  first 
step,  deccmpiosing  A into  LG  . 

The  decomp^osition  of  A proceeds  by  means  of  row  op^erations.  A row 
operation  consists  of  adding  a multiple  of  one  row  of  A to  anotlier  row 
of  A . If  the  multiple  is  chosen  correctly,  the  modified  row  will  have 
a zero  in  a previously  non-zero  position.  By  systematically  aijlying 
such  row  operations,  one  can  transform.  e original  matrix  A into  an 
upper  triangular  matrix  U ; the  row  operations  performed  define  a l.iwor 
triangular  matrix  L such  that  LU  = A , 
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If  A is  originally  a dense  (mostly  non-zero)  matrix,  then  LU 

2 ^ 

decomposition  requires  0(n  ) space  and  0(n'^)  time,  and  frontsolving 

2 

and  backsolving  require  0(n  ) time.  In  many  large  systems  of  equations, 
however,  the  matrix  A is  sparse.  For  a sparse  matrix,  the  time  and 
storage  space  required  by  Gaussian  elimination  depend  in  a complicated 
way  upon  the  zero-non-zero  structure  of  the  matrix.  In  particular,  a 
row  operation  may  Introduce  new  non- zeros  (called  fill-in)  into  positions 
originally  zero.  It  is  desirable  to  rearrange  the  matrix  A by  means 
of  row  and  column  permutations  so  that  the  fill-in  and  running  time  of 
Gaussian  elimination  are  reduced. 

For  this  lurp-ose  it  is  useful  to  represent  the  zero-non-zero  structure 
of  A by  a graph  G . The  graph  contains  one  vertex  for  each  row  and 
col'Limn  of  A and  one  edge  (i,j)  for  each  non-zero  entrj'  (i,j)  in  A , 

If  A is  sjTiimetric,  G is  'jndirected;  if  A is  unsynimetric,  G is 
directed.  The  g’-aph  G represents  A and  all  matrices  formed  by 
simultaneously  jiermuting  rows  and  columns  of  A . By  studying  the  properties 
of  G , it  may  be  possible  to  find  a reordered  version  of  A such  that 
Gaussian  elimination  is  efficient,  (it  is  necessary  to  know  that  the 
permutations  do  not  destroy  the  numeric  stability  of  the  elimination 
process.  1 shall  ignore  this  issue  here;  see  Forsythe  .and  Moler  [1967], 
Tewarson  [ 1973 ] • ) 

Farter  [19<^1]  was  one  of  the  first  to  suggest  the  usefulness  of  this 
approach.  The  idea  has  been  extensively  developed.  For  general  results 
concerning  the  relationship  between  Gaussian  elimination  and  graph  theory, 
see  Rose  (I97O];  Harary  [1971];  Rose  [1975];  Bose,  Tarjan,  and  Lueker 
[197tjj:  Duff  [1970];  and  Rose  and  Tarjan  [I977]. 

[Figure  5.2] 
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As  £in  exanii'le  of  the  improvement  possible  by  taking  advantage  of 

sparsity,  consider  the  graph  in  Figure  5.2.  Such  a kxk  grid  gra^h 

arises  in  the  numeric  solution  of  differential  equations.  Ordinar;/ 

2 ^ 

dense  Gaussian  elimination  requires  0(n  ) space  and  O(n^)  time  on 

2 

such  a matrix,  if  n = k . The  bandwidth  scheme  of  sparse  elimination 

'T  Jr)  ^ 

reduces  the  space  to  0(ir '‘■)  and  the  time  to  0(n  ) (Cuthill  and 
McKee  [l''*’9]>  Tewarson  [1S>7?]''.  George  [1975]  discovered  an  even  better 
metlr'd,  called  nested  dissection,  iviiich  requires  0(n  log  n)  S3‘ace  and 

7 - • 

’(n  *')  time.  Hoffman,  Martin,  .and  Rose  [1975)  showed  that,  to  within 

a constajit  factor,  nested  dissection  requires  the  least  fill-in  and 
comj'uting  time  of  £iny  ordering  scheme  for  Gaussian  elimination  on  kxk 
grid  graphs. 

Nested  dissection  is  a recursive  method  vdiich  uses  the  f.act  that  a 
(2k+l)  X (2k<-l)  grid  grapii  consists  of  four  kxk  grid  gra}lu'  and  the 
l*k+l  -vertex  boundary  between  them  (Figure  5.2).  Many  sp.arse  matrices 
which  arise  in  jiractice  do  not  have  such  a nice  structure,  and  one  m.ight 
ask  wiiether  nested  dissection  has  any  natural  generalization.  Recentlv 
Lipton,  Rose  :md  Tarjaii  (Tarjjui  [I'tyib])  discovered  a way  to  e.’rtend  neste 
dissection  to  arbitrary  jianar  grapihs  so  that  the  storage  s]  ace  is  still 
0(n  log  n)  and  the  running  time  still  0(n  ' ) . Such  gra^ihs  ai-ise  in 
twii- LLmensional  finite  element  jiroblems  (Martin  and  Carey  [197^]'. 

Global  Flow  Analysis. 

Dystemr  of  linear  eqtiations  arise  in  contexts  otlier  than  linear 
algebra.  For  instance,  the  shortest  path  problem  can  be  formulated  as  a 
system  of  equations,  witli  minimization  replacing  additi.-.n  and  add'tion 
rejjlacing  multijilication  (Backhouse  and  Carr4  [1975]).  Another  situation 


whert.’  systems  of  linear  equations  occur  is  in  the  global  flow  analysis 
of  computer  j'rcgrams.  Suppose,  for  instance,  that  we  wish  to  modify  a 
computer  program  so  that  it  does  not  recompute  an  expression  unless  the 
value  of  one  of  the  variables  in  the  expression  has  chfinged. 

The  first  step  in  the  analysis  is  to  represent  the  program  by  a 
flow  grapih.  I'Jach  vertex  in  the  flow  graph  rep^resents  a basic  block  of 
the  pirogram  (a  set  of  program  statements  having  a single  entry  p'oint  ;md 
a single  exit  pioint).  tech  edge  in  the  flow  graph  represents  a transfer 
of  control  I'rom  one  basic  block  to  another.  The  problem  of  deteimiining, 
for  each  basic  block,  the  set  of  available  expressions  (those  wh: ch  do 
not  need  to  be  recomputed)  caji  then  be  formulated  as  a system  of  linear 
equations  with  one  variable  for  each  basic  block.  The  variable  is  a bit 
vector,  with  one  bit  corresponding  to  each  program  expression,  and 
appj-opriate  bit  vector  opierations  replace  addition  and  multipdi cation  in 
the  system  of  equations.  The  sparsity  stnjcture  of  the  matrix  corresponds 
to  the  flow  graph  representing  the  prograiu.  For  further  details  of  this 
correspjondence,  see  Kildall  [1973  ]>  Schaefer  [1975]>  aiid  Allen  and  Cocke 
[17f6]. 

One  can  use  standard  Gaussian  elimination  techniques  to  compute 
available  expressions,  but  it  is  often  useful  to  take  advantage  of 
spfiTsity.  Tlae  flow  grapihs  of  many  comp'uter  programs  have  a sp>ecial 
property  called  reducibility,  which  means  in  essence  that  every  cycle  has 
a single  entry  point  from  the  starting  block  of  the  program.  Allen  [197O] 
and  Cocke  [1970]  first  formulated  this  notion  of  reducibility,  p'resented 
an  0(nm)  -time  algoritlim  to  test  for  reducibility,  and  used  tills  test  in 
an  0(nm)  -time  algorithm  for  global  flow  analysis  of  reducible  graphs. 
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llol'croft  and  Ullman  [1^7^2j  discovered  ;m  0(m  log  n)  -time  test  for 
reducibility,  which  Lfllman  [1973]  combined  with  clever  use  of  2-5  trees 
to  give  an  0(m  log  n)  -time  method  for  global-  flow  analysis.  Kennedy 
[1973]  discovered  a rather  different  method  for  global  flow  analysis, 
vdiicli  is  0(m  log  n)  -time  by  a result  of  Aho  and  Ullman  [ig75l.  GraJiam 
and  Wegmati  (IQ'/t  ] discovered  how  to  use  j:ath  compression  to  get  yet 
another  0(m  log  n)  -time  algoritVim.  Tarjan  [l''',  ltcj  gave  an  0(m  u:(m,n'!) 
-time  algoi'iflmi  for  testing  reducibility,  and  later  imp^roved 
the  Gr;diam-Wegman  algorithm  to  run  in  0(m  a(m,  n))  time  (Tarjan  [19'/’5c]). 
Here  a(m,  n)  is  a limctioiuil  inverse  of  Ackermann's  function  t ' be  defined 
below. 


Fatteni  Matching  on  Strings. 

Suppose  X ;ind  y are  two  strings  com.piosed  of  characters  selected 
from  a finite  alphabet,  and  we  wish  to  detennine  wliere  x occui’s  as  a 
contiguous  substring  of  y . If  m is  the  length  of  x and  n is  the 
length  of  y , then  a straightforward  algorithm  solves  this  problem  in 
0(nm)  time.  Knuth,  Morris,  and  Pratt  [I't/’Yl  devised  an  O(ntm)  -time 
algorithm  for  p-attern  matching.  Their  filgoritiim  jirocesses  the  pattern  x , 
creating  a data  structure  representing  a p^rogram  to  recognize  the  p>attern. 
Tiie  algorittim  then  scans  the  string  y character-by-char’acter  according 
to  the  steps  of  the  program.  Boyer  and  Moore  [ 1975 ] proposed  an  oven 
better  algoritlim,  which,  althougli  it  has  an  0(n*m)  running  time  in  tlie 
worst  case  (Knutii,  Morris,  and  Pratt  [l<^r/2]),  requires  only  o(n(l>ig  mi  mi 
time  on  tlie  average,  wliere  q is  the  olpliabet  size. 

A generalization  ol'  the  pattern  matching  problem  is  to  find  Uie  longest 
common  contiguous  substring  of  two  strings  x and  y . llie  piattern 


matching  algoritlims  mentioned  above  do  not  seem  to  apjly  to  this  problem. 
Karp,  Miller,  and  Rosenburg  [1972]  described  an  0((m+n)  log(nv+-n))  -time 
algoritlmi  for  longest  common  substrings.  Weiner  [l',/75l  discovered  an 
algorithm  using  trees  in  a new  way  wliich  solves  the  j^roblem  in  0(n+m) 
time.  McCrcight  [197<j]  has  } rovided  a oi;nplification  and  clean  descri]t;on 
of  this  algorithm. 

Strong  Components. 

The  strong  components  problem  is  to  determine  the  strongly  connected 

components  of  a given  directed  graph  with  n vertices  and  m.  edges. 

This  p?roblem  occurs  in  finding  the  irreducible  blocks  of  a non-syi;imetric 

matrix  (Forsythe  and  Moler  [19b?  ])>  '-ri  finding  ergodic  subchains 

and  transient  states  of  a Markov  chain  (Fox  and  Landy  [I968]),  and  in 

finding  the  transitivity  sets  of  a set  of  permutations  (McKay  and  Regener 

2 

[197^]').  fargent  and  Westerberg  [19ot]  gave  an  0(n  ) -time  algorithm. 
Ntunro  [1971]  described  an  improved  algorithm  with  a running  time  of 
0(n  log  n+m)  . Tarjan  [1972]  presented  an  0(n+m)  -lime  algorithm 
which  uses  depth-first  search  ;ind  a few  simple  data  structures  to  solve 
this  problem. 

Planarity  Testing. 

Let  G be  a graph.  The  j'lanarity  testing  problem  is  to  determine 
whether  G can  be  drawn  in  a plane  so  that  no  two  edges  cross. 

Kurakowski  [1950]  provided  an  elegant  mathematicjil  characterisation  of 
planar  graphs,  showing  that  a graph  G is  planar  if  and  only  if  it  does 
not  contain  one  of  the  two  graphs  in  Figure  5.5  as  a generalised  subgraph, 
unfortunately,  Kuratowski ' s criterion  seems  to  be  useless  as  a practical 
test  for  planarity.  Auslander  and  Farter  [19^'l]  proposed  an  algorithm 


which  tests  planarity  by  trying  to  construct  a planar  representation 
I’or  the  graph.  They  gave  no  time  bound  for  the  algorithm,  and  their 
presentation  contains  an  error;  the  proposed  algorithm  may  run  forever, 
doldstein  [I965]  correctly  formulated  this  algorithm,  and  Shirey  [19^9] 
gave  an  0(n^ ) -time  implementation  of  it.  Hopcroft  and  Tarjan  [1S^72] 
t-'mbined  dejth-first  search  and  aj-proj'riate  data  structurer 
in  an  0(n  log  n)  -time  implementation,  vhich  was  later  siiajlified  and 
improved  to  0(n)  (Hopcroft  and  Tarjan  [197*+]  )• 

[Figure  5.3] 

Lempel,  Kvcn,  and  Cederbaum  [I967]  presented  another  good  algorithm, 
witiiout  giving  an  explicit  time  bound.  Their  algorithm  can  easily  be 
implemented  to  run  in  0(n  ) time.  Booth  and  Lueker  [197*-]  chewed  how 
to  use  their  P-Q  tree  data  structure  in  an  0(n)  -time  im.plementation  of 
the  algoritiuri. 

Mascimum  Network  Flow. 

Let  G be  a directed  graph  with  two  distinguished  vertices,  a 
source  s and  a sink  t . For  each  edge  e in  G , let  c(e)  be  a 
non-negative  real-valued  capacity.  A flow  f on  G is  a non-negative 
value  f(e)  on  each  edge  such  that,  for  all  vertices  v except  s and  t , 
the  total  flow  on  edges  entering  v is  equal  to  the  total  flow  leaving  v . 
The  value  of  the  flow  is  the  total  flow  leaving  s (which  is  equal  to 
tlie  total  flow  entering  t ).  The  maximum  network  f_ow  problem  is  to 
determine  a flow  f(e)  of  maximum  value  satisfying  f(e)  < c(e)  for 
all  edges  e . 

Classic  work  by  Ford  and  Fulkerson  [I962]  produced  an  elegant  algorithm 
wh-Lch  augments  flow  along  paths.  Uhfortunately,  in  the  worst  case  their 


r 

i 
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algorithm  requires  exijonential  time,  though  it  works  exceedingly  well 
in  practice.  Edmonds  and  Karji  [I972],  by  using  breadth- I’irst  search 

O 

to  guide  the  selection  of  augmenting  paths,  produced  an  0(iun"')  -time 
variation  of  the  Ford-Fulkerson  algorithm.  Independently,  Dinic  [I97O] 
used  breadth-first  search  p^lus  impiroved  up>dating  methods  to  achieve  an 

O 

O(n^m)  time  bound.  Tlie  best  algorit)im  so  far  found  for  this  problem  is 
due  to  Karzanov  [197^],  '>dio  imp  roved  Dinic' s algorithm  to  obtain  an 
O(n^)  time  boiuid. 

Grap'h  Matching. 

If  G is  an  undirected  grapli,  the  graph  matching  problem  is  to  find 
a naximim  number  of  edges  in  G , no  two  having  a common  endpoint.  Such 
a set  of  edges  is  a maximum  matching.  An  important  special  case  of  this 
problem  is  its  restriction  to  bipantite  graphs.  A grapli  is  bipfirtite  if 
its  vertices  can  be  partitioned  into  two  sets  so  that  no  edge  connects 
two  vertices  in  the  same  set. 

Tlie  bipartite  graph  matching  problem  can  be  transformed  via  a 
linear-time  algorithm  into  a network  flow  problem  in  wiiich  all  edge 
capacities  are  one  (E’ord  and  Fulkerson  [I962]);  for  sucli  a problem,  tlie 
Ford-Fulkerson  algoritlim  has  an  O(nm')  time  bound.  Kuhn  [ly^'j]  used 
results  of  Egervary  to  obtain  essentially  the  same  alg.'r'tiK;,  called  the 
Hungarian  method.  Hopcroft  and  Kaiq  [17!3]  used  breadth-first  searcli 
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and  improved  updating  methods  to  achieve  an  0(n  m)  time  boutid. 
Their  algoritlim  is  essentially  the  same  as  Dinic' s (EVen  and  Tarjan 
fl975]). 
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thoufiii  a g >ou  itJ^orltiun  dua  nil  Toll  w fror.  their  resultr.  Edmonds  [19tj5] 

used  cycle  slirinking  jlus  the  auf?’:enting  paths  idea  to  give  a polynomial- 

time  algorithm.  He  claimed  an  0(n  ) time  bound,  though  it  is  not  hard 

2 

t.  implement  Edmonus'  lilgorithm  to  run  in  0(n  m''  time.  Lawler  [I976J 

and  'Jabow  [19’*  ] indejiendently  gave  o(nm)  -time  algorithms.  LVen  and 
Kariv  [igr^5]  iiigeniously  co»nbined  the  ideas  of  Hojcroft  and  Karp  and  the 

,1/2  . 

data  structures  of  labow  to  obtain  an  0(n  ' ra  log  n)  -time  algorithm. 

Set  Union. 

Let  Sg,  . . .,  be  n disjoint  sets,  each  containing  a single 
element.  Hie  disjoint  set  union  problem  is  to  carry  out  a sequence  of 
operations  of  the  following  two  types  on  the  sets. 

find(x)  determine  the  name  of  the  set  containing  element  x . 

union (A, B) : add  all  elements  of  set  B to  set  A (destroying 

set  B ). 

The  operations  are  to  be  carried  out  on-line;  that  is,  each  instruction 
must  be  completed  before  the  next  one  is  known.  Assume  for  convenience 
that  the  sequence  of  operations  contains  exactly  n-1  union  operations 
(so  that  after  the  last  union  all  elements  are  in  one  set)  and  m > n 
intermixed  find  operations  (if  m < n , some  elements  are  never  found). 

Galler  and  Fischer  [I96U]  proposed  an  algorithm  for  this  problem  in 
which  each  set  is  represented  by  a tree.  Each  vertex  of  the  tree  represent 
one  element.  The  root  of  the  tree  contains  the  name  of  the  set,  and  each 
tree  vertex  has  a pointer  to  its  pfirent  in  the  tree.  See  Figure  5-^. 

A find  on  element  x is  performed  by  starting  at  the  vertex  representing 

x and  following  piarent  pointers  until  reaching  the  root  of  the 

tree;  this  root  contains  the  set  name.  A uni  ”!  of  sets  A and  B is 


performed  by  maMng  the  root  of  the  A tree  the  parent  of  the  root 
of  the  B tree. 

[Figure  5.U] 

This  algoritlm  requires  0(nm)  time  in  the  worst  case,  since  an 
unfortunate  sequence  of  unions  can  build  up  a tree  consisting  of  a single 
long  path.  Caller  and  Fischer  modified  the  union  procedure  in  the 
following  way;  if  B contains  more  elements  than  A , then  the  root 
of  the  B tree  is  made  the  parent  of  the  root  of  the  A tree,  and  the 
name  A is  moved  to  the  old  root  of  B . See  Figure  5 •5*  This  weig'nted 
union  heuristic  improves  the  algorithm  considerably;  Caller  and  Fischer 
proved  an  0(m  log  n)  time  bound. 

[Figure  5.5] 

Mcllroy  and  Morris  (Aho,  Hopcroft,  and  Ullman  [1517^1]''  modified  the 
find  procedure  by  adding  a heuristic  called  path  compression:  after  a 

find  on  element  x , all  vertices  on  the  path  from  x to  the  root  are 
made  children  of  the  root.  See  Figure  5.c.  This  increases  the  time  of 
a find  by  a constant  factor  but  may  save  time  on  later  finds. 

[Figure  5.-  ] 

The  set  union  algorithm  with  path  compression  is  very'  easy  to  program 
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but  very  hard  to  analyze.  Fischer  [1972]  proved  an  0(mn  ' ) upper  hound 
and  an  Q(m  log  n)  lower  bound  on  the  worst-case  running  time  of  the  algorithm 
with  path  compression  but  without  weighted  union.  Paterson  [1972]  improved 
the  upper  bound  to  0(m  log  n)  and  thus  determined  the  running  time  to 
within  a constant  factor  for  the  case  when  m is  0(n)  . With  both  path 
compression  and  weighted  union,  the  algorithm  is  even  harder  to  analyze. 

Fischer  [1972]  proved  an  0(m  log  log  n)  upper  bound  on  the  running  time. 


H-.'i  croft  aud  LIlLirian  [\^T3\  iinj.  roved  the  upper  bound  to  0(m  log  nl  , 

1 times 

. ^ ^ 

where  log  n = min{i  | log  log  ...  log  n ^ 1]  . Tarjan  [19]5si]  imj  roved 
the  upper  bound  to  0(n  a(m,  n))  , where  .'{(m,  n)  is  a 
functional  inverse  of  Ackermann' s {’unction  (AcKermaiin  (r,128])  defined 
as  follows. 

For  i,  j > 0 let  the  function  A(i.J)  be  defined  by- 

Co.!)  A(;,0)  = 0 ; 

A(  = 2*^  for  j > 1 ; 

A(i.l)  = A(i-1,2)  for  i > 1 ; 

A(i.j^  = A(i-1,A(; . j'-l))  for  i > 1 . j 1 2 . 

Let 

(‘j.2!  n'  = min[:  > 1 j A(i  , ^ n}  •- 

The  bound  0(m  i(m,n))  is  a rather  complicated  one  for  such  a simple 
algorithm.  One  may  naturally  ask  whether  it  is  im^rovaole.  Tarjaji  [1975a] 
showed  that  there  are  w rsi-case  instances  ol'  the  set  'union  jroblem 
which  require  n(m  ,t(m,n)'  time  wiien  solved  by  the  jath  comj-ress: on 
alg  rithm.  In  fact  smy  linked  memory  machine  reqiiires  .j(m  ,t(m,n)) 
time  in  the  worst  case  to  solve  the  set  uaion  problem.  (Tarjan  [1''^',']  . 

Thus  Ackermann' s function  is  inherent  in  the  problem. 


For  any  real  number  x . |_xj  denotes  the  greatest  integer  n -t  larger 
than  X . 


Future  Directions. 


C'. 

The  field  of  combinatorial  algorithms  is  too  vast  to  cover  in  a 
single  paper  or  even  in  a single  book.  I have  tried  zo  point  out 
some  of  the  major  results  and  underlying  ideas  in  this  field,  but  there 
are  certainly  many  important  results  I have  had  to  omit.  Though  much 
w rk  on  combinatorial  algsrlthms  ha.":  been  done,  much  remains  to  be  done. 
In  this  concluding  section  I would  like  to  saggest  five  areas  for  future 
research,  areas  in  wiiich  relatively  little  work  has  been  done  but  in 
which  the  rewards  are  j)otentially  great. 

Is  p = 

Answering  this  question  would  be  a major  breakthrough  in  complexity 
theory.  Althougli  many  people  have  attempted  to  solve  this  problem,  very 
little  pr -gress  has  been  made.  It  seems  that  some  ma^’or  new  ' ica  is 
n'^eded;  the  evidence  of  Baker,  Gill,  and  Solovay  [191'5]  suggests  that 
d i agonal  i sat  ion,  the  standard  technique  for  proving  problems  riard,  may 
not  be  power  till  enough  to  sh  w p ^ 7)p  . It  is  even  conceivable  that 
the  p = 7?P?  problem  cannot  be  solved  within  the  framework  jf  foim.al 
set  theory'  (Hartmanis  and  Hopcroft  [19'/’  ])• 

M re  generally,  we  now  know  almost  nothing  about  the  relative  p^ower 
deterministic  and  non-determini stic  algorithms,  and  about  the 
relationship  between  time  and  space  as  measures  of  comp^lexity.  Any 
results  in  this  area  would  be  important.  Recently,  Hopcroft,  Paul,  and 
Valiant  (19’(’5l  were  able  to  show  that  any  computation  'requiring  i.(n) 
time  on  a multitape  Turing  machine  can  be  carried  out  in  t(n)^log  t(n) 
spaoe.  Thus,  at  least  for  multitape  Turing  machines,  space  is  a more 
Valuable  resource  than  time.  This  is  the  only  such  result  known. 
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One  ai  I roach  to  the  p = 7(5^?  question  is  to  consider  for  a particular 
7/p-conplete  problem  a restricted  class  of  algoritluns  and  to  show  that 
evei';,-  algoritlur.  in  this  limited  class  requires  more  than  polynomial  time. 
Results  of  this  kind  have  been  obtained  for  the  satisfiability  problem 
(Galil  fhe  maximum  stable  set  problem  (Cbvatal  [19^6]),  and  the 

graph  coloring  problem  (McDiar.nid  [L'lro  ]'. 

Another  approach  is  to  consider  models  oi’  computation  other 
than  Turing  machines.  One  pv^ssibility  is  to  study  the  size  of  Boolean 
circuits  for  computing  Boolean  fu>ictlon_.  For  results  in  this  area, 
see  Savage  [li'iT’  ].  A related  model  is  the  pebble  game  used  by  Hopcroft, 
laul,  and  Valiant  to  obtain  their  time-space  tradeoff  resxilt.  A study 
■■  Boolean  circuits  and  the  pebble  ga;ne  leads  rapidly  to  unanswered 
c /tnbinat  'r ial  questions  (Valiant  [19T5b,  197^]). 

Average-Case  Analysis. 

Alth  ugh  most  of  the  wjrk  on  combinatorial  algorithms  outside  the 

areas  of  s -ting  and  searcning  has  oeen  wjrst-case  analysis,  average-case 

a:ialysis  is  p Jtentially  important  and  useful.  The  results  of  ErdBs  and 

Renyi  .nid  otners  on  random  graphs  form  a starting  place  for 

aver,'jge-case  an-v.ysis  of  graph  algorithms,  Spira  [1975]  has  devised 

an  0(n  (log  n V"  aver-ige  time  algorithm  for  the  all-pairs  shortest 

path  problem,  wiiich  Bloniarz,  Fiscner,  aiid  Meyer  [1976]  modified  to 

2 

compute  transitive  closures  in  0(n  1 >g  n)  average  time.  Schnorr  [1977] 

has  devised  an  0(n  log  n h m'  average  time  transitive  closure  algorithm. 

Yao  [ig^p.’],  Doyle  and  Rivest  [l?p  ],  and  Knuth  and  SchBnhage  [1977]  have 
analyzed  the  behavior  of  set  union  algorithms  for  several  probab'lity 
distributions.  Much  more  work  in  this  a^ea  is  needed. 

LB 


Gill  [1‘17^]  Rabin  [I976]  have  proposed  another  kind  of  aver.-ige- 


case  model  of  complexity,  in  which  the  algorithn;  makes  use  of  random 
choices.  For  such  an  algorithm,  one  may  be  able  to  say  that  the  .'algorithm 
runs  fast  on  the  average  independent  of  the  input  distribution,  because 
the  average  is  taken  not  over  the  input  but  over  the  possible  comjutations 
of  the  algorithm  on  a given  injut.  Algorithms  exist  for  testing  primality 
(Strassen  and  Solovay  [1977],  Rabin  [19’/o]),  finding  closest  points 
(Rabin  [197c]),  and  hashing  (Carter  and  Wegman  [19771)  which  are  good 
in  this  sense. 

Constant  Factors  and  Algorithm  Trade-Offs. 

The  choice  of  an  algorithm  in  practice  may  defend  upon  more  than 
asymptotic  running  time.  A simple  algorltlim  may  be  better  on  intermediate- 
sized problems  than  a more  conrplicated  algorithm  with  a faster  asymptotic 
running  time  but  a larger  constant  factor.  If  two  algori thr’.r  have  the 
same  asymi  totic  running  time,  then  the  constant  factors  may  govern  the 
choice  between  them.  More  careful  rinalysis  to  determine  constant  factors 
and  trade-offs  between  algorithms  would  be  a useful  contribution  to 
practical  comp-uting.  Such  analysis  for  a number  of  problems  appears 
in  Knuth's  books  (Knuth  [19^'8,  19<.  9,  l'J7?])-  A recent  example  of  this 
research  is  work  by  Brown  [1977],  wiiich  compares  implementations  of 
priority  queues  and  suggests  that  the  binomial  tree  representation  is 
best  in  most  circumstances. 

Low-Order  Lciwer  Bounds. 

Just  as  little  is  known  ,-ibout  the  boundary  between  tractable  ,tnd  intractable 
pr-,blems,  little  is  known  about  wiietVier  or  not  the  existing  good 
algorith.ms  are  imp  r'Tvable.  Figure  5*1  suggests  several  tantalizing  questi  >ns. 
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>1  CaJi  twj  matrices  be  multiplied  in  less  than  0(n’"  ) time?  Ca^i  the 

discrete  Fourier  transfonn  be  computed  in  less  than  0(n  log  n)  time? 

Can  maximum  network  flows  be  found  in  less  than  O(n^)  time?  IJun-i inear 
lower  bounds  exist  for  only  a '’ew  I'roblemS;,  ouch  ac  sorting  (Knuth  [lSf75]). 
finding  shortest  paths  (Yao,  Avis,  and  Rivest  [ 1977 ] \ disjoint  set  union 
(Tarjan  [I'VY]'')  and  evaluation  of  symmetric  I'unctions  (Strassen  [197^]). 

General  Iroperties  of  Data  Structures  and  Basic  Methods. 

The  range  of  techniques  and  algorithms  outlined  in  Sections  1-  and  9 
suggests  a basic  qaestion:  Confronted  with  a problem,  how  does  one 

construct  a good  algorithm  for  it?  Is  there  a "calculus  of  data  structures" 
by  which  one  can  choose  the  appropriate  data  representation  and  techniques 
•'  r a given  j-roblem?  What  makes  one  data  structure  better  tliaji  another 
I'or  a certain  application?  The  known  results  cry  out  for  an  underlying 
thesry  to  explain  them.  This  is  perhaps  one  of  the  most  challenging 
pr  blems  facing  researchers  in  algorithmic  complexity. 
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A]']  ondix : 


Te  mi  no  logy  (See  also  Berge  [19<'2],  Busacker  'ind  Saaty  [I965], 


and  Harary  [ I969 1 ) . 

A g-'a]  h G = (V,  E)  ic  an  ordered  pair  c-nsi sting  of  a set  V of 
vertices  and  a set  E of  e(,4;es.  Either  the  edges  are  ordered  pairs  (v,  w) 
of  distinct  vertices  (the  grajh  is  directed).,  or  the  edges  are  unordered 
pairs  of  distinct  vertices,  ;ilso  respresented  as  (v, w)  (the  graph  is 
undirected) . If  (v,  w)  is  an  edge,  v anu  w are  its  endj.ioints  and 
are  adjacent.  The  edge  (v,  w)  leads  fr.Tn  v to  w , (if  undirected, 
the  edge  also  leads  from  w to  v A graph  G'  = (V’,E')  is  a subgraph 
of  G if  V c V and  r.'  c E , G'  is  si anni ng  if  V = V . A graph 

G’  = (V  , E'  ) is  a homomorp.il ic  image  of  G if  there  is  a maiping  from  V 
onto  V such  that  (x,y)  c E'  if  and  only  ii'  x = f(v)  and  y = f(w) 

for  some  (v.w)  e E , G and  G'  are  Isomori  hie  if  the  mapp  ing  is 

one-to-  ne.  A graph  G'  = (V”,E’)  is  a generalised  subgraph  of  G if 
G'  is  a subgrapii  of  a homomorphic  image  of  G . 

A p ath  from  v^  to  v^^  in  G is  a sequence  of  edges  (v^^,  v^),  . . . , (v^ 
This  path  is  raid  to  con'*  ain  edges  (v^,  Vg),  . . . , (v^  l’''''n^  vertices 

v^,,..,v^  , :md  to  avoid  all  other  edges  and  vertices.  The  path  is  simp le 
if  distinct  except  possibly  v^  and  v^  ; the  path  is  a 

cycle  if  Vj^  = v^  . Tlie  transitive  closure  of  G = (V.  , ' is  the  graph 

G*  = (V,  E^)  such  that  (v,  w)  c E^  if  and  onlj'  if  v ^ w and  there  is  a 

path  from  v to  w in  G . 

An  undirected  graph  is  connected  if  there  is  a path  I’rom  any  vertex  to 
any  other  vertex.  A directed  graph  is  strongly  connected  If  there  is  a 
path  from  .-iny  vertex  to  any  other  vertex.  Tlie  maximum  connected  (strongly 
connected)  suDgrajhs  of  a graph  are  called  its  connected  components 
(ctr  .ngly  cc.nnectod  comp  snents) . A graph  is  planar  if  it  can  be  tiravm  in 
th“  pl'Uie  (with  vertices  as  jwintc  and  edges  as  simple  curves)  so  that  no 


two  edges  intersect  excejt  at  a conanjn  endpoint. 


A tree  T is  a connected,  undirected  graph  which  contains  no 
cycles.  In  a tree  there  is  a unique  simple  path  between  any  pair  of 
distinct  vertices.  A rooted,  undirected  tree  (T, r)  is  a tree  with  a 
distinguished  vertex  r , called  the  root.  A rooted,  directed  tree  is  a 
directed  graph  T with  a unique  vertex  r such  that 

(i)  there  is  a path  from  r to  any  other  vertex; 

(ii)  each  vertex  except  r has  exactly  one  edge  leading  to  it; 

(iii)  r has  no  edges  leading  to  it. 

Any  rooted,  undirected  tree  (T, r)  can  be  converted  into  a rooted, 
directed  tree  by  directing  each  edge  (v, w)  so  that  v is  contained  in 
the  ] ath  fr  -m  r to  w . 

In  a rooted,  directed  tree,  a vertex  w is  a descendant  of  a vertex 

V (v  is  an  ancestor  of  w ) if  there  is  a path  from  v to  w . A vertex 

V is  a child  of  v (v  1-  the  | arcnt  of  w ) if  (v,  w)  is  an  edge  in  the 

tree.  These  definitions  extend  to  rooted,  'undirected  tree  by  directing  the 

edges  of  the  tree  as  above.  If  G is  a graph,  a spanning  tree  of  G is  a 

rooted  tree  wiiich  is  a scanning  subgraph  of  G . 

A partition  of  a set  S is  a collection  of  subsets  S , S , ...,S, 

-L  ^ X 

k 

of  S such  that  U S.  = G and  S.  flS.  = 0 if  i * j . If  p” 

J=1  ^ - J 

ai’e  partitions  of  S , ^ is  a refinement  of  y (and  y is  a 

coarsening  of  ^ ) if,  for  all  e , there  is  some  y^  e y such  that 
j.  c.  J.  , 
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Table  ~ .2.  Maxiinuni  Size  of  a Solvable  I'roblem. 

(a  factor  of  ten  increase  in  machine  speed  correc]  onds  to 
a factor  of  ten  increase  in  time.) 
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Data  structui’es  (built  from  arrayr  and  linkc-d  rtructu.re.' ' . 

a.  Lists. 

b.  Unordered  sets, 

c.  Ordered  sets. 

d.  Graphs. 

e.  Trees. 

Recursion. 

a.  Dyiiasnic  ; rjgramrdng. 

Graph  searching. 

a.  Depth- first. 

b.  Breadth- fi rst . 

i.  Shortest-first. 

ii.  Lexicographic, 

Optimization  methods. 

a.  Greed. 

b . Augment  at i on . 

Data  uj'dating  met, beds. 

a.  lath  cj.mrression. 

b.  Partition  refinement. 

c.  Linear  arrangement. 

Graph  mapping. 

a.  Decomposition  (by  subgraplis ) . 

b.  Shrinking  (by  graph  homomorphism''. 


Table  U.l,  Techniques  for  Good  Algorithms. 
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Figure  1.1.  A linked  structure  and  its  representation  by  arrays. 

(a)  Record  format. 

(b)  Linked  structure. 

(c)  Representation  by  three  arrays. 
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head  • 


tail  — I 


Figxire  l-.J.  Representation  of  ! 

linked  structure. 
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Figure  1^.1+.  Representation  of  a graph, 

(a)  Graph. 


(b)  Adjacency  matrix. 

(c)  Adjacency  structure. 


(b) 

Figure  H.8.  Breadth-first  search.  Level  indicated  in  jarentheses. 

(a)  Search  of  graph  in  Figure  1.'  . 

(b)  Search  of  graph  in  Figure  I4.7. 
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1.  Discrete  Fourier  transform  (DFT) : 

recursion. 

2.  Matrix  multiplication  (MM); 

recursion, 

3.  Linear  equations  on  a planar  graph  (LEG): 

recursion,  decomposition  by  connectivity,  breadth-first  search. 
Global  flow  analysis  (GFA): 

decomposition  by  connectivity,  path  compression,  depth-first  searcli. 
5 . Pattern  matching  on  strings  (PM) : 
data  stnicturos . 

' . Strong  components  (SC): 
depth-first  search. 

7.  Planarity  testing  (PT): 

depth- first  search. 

8.  Maximum  network  flow  (MTIF): 

augmentation,  breadth-first  search. 

9.  Graph  matching  (ffl-l) ; 

augmentation,  breadth-first  search,  cycle  shrinking. 

10.  Set  union  (SU); 

path  compression. 
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Fxgiire  5.1.  Recent  complexity  improvements. 

n = size  (number  of  vertices  in  graph  problems), 
m = second  parameter  (number  of  edges  in  graph  problems). 


union (a, B) : 


Figure  5.5-  Implementation  of  union. 


(^'  BaaLc  method. 

(b)  Weighting  heuristi 


